Giters

intel / ipex-llm

Accelerate local LLM inference and finetuning (LLaMA, Mistral, ChatGLM, Qwen, DeepSeek, Mixtral, Gemma, Phi, MiniCPM, Qwen-VL, MiniCPM-V, etc.) on Intel XPU (e.g., local PC with iGPU and NPU, discrete GPU such as Arc, Flex and Max); seamlessly integrate with llama.cpp, Ollama, HuggingFace, LangChain, LlamaIndex, vLLM, DeepSpeed, Axolotl, etc.

Repository from Github https://github.comintel/ipex-llmRepository from Github https://github.comintel/ipex-llm

Documentation for running ipex-llm (at least Ollama) is incorrect and no device will be found

anteew opened this issue · comments

commented

I found this issue on Ubuntu 24.10, if you are on a different version YMMV.

These instructions ultimately end up not working:

https://github.com/intel/ipex-llm/blob/main/docs/mddocs/Quickstart/bmg_quickstart.md

after running through the instructions start-ollama.sh (from the portable version) throws a golang error immediately upon starting.

clinfo shows essentially nothing, no devices:

clinfo
Number of platforms 0

ICD loader properties
ICD loader Name OpenCL ICD Loader
ICD loader Vendor OCL Icd free software
ICD loader Version 2.3.2
ICD loader Profile OpenCL 3.0

sudo clinfo does show the devices.

Something in the process in the bag_quickstart.md adds environment variables (3 of them):

OCL_ICD_VENDORS_RESET=1
OCL_ICD_VENDORS=/home/ubuntu/miniforge3/envs/battlemage/etc/OpenCL/vendors
OCL_ICD_FILENAMES_RESET=1

ls /home/ubuntu/miniforge3/envs/battlemage/etc/OpenCL/vendors

intel-cpu.icd
intel-fpga_emu.icd

cat intel-cpu.icd

/opt/anaconda1anaconda2anaconda3/lib/libintelocl.so

Everything after /opt just doesn't exist on my system.

unset OCL_ICD_VENDORS_RESET
unset OCL_ICD_VENDORS
unset OCL_ICD_FILENAMES_RESET

Seems to fix the issue.

What is inserting this: /opt/anaconda1anaconda2anaconda3/lib/libintelocl.so into this file: /home/ubuntu/miniforge3/envs/battlemage/etc/OpenCL/vendors/intel-cpu.icd ?

Now I see this when I run ./start-ollama.sh & ./ollama run deepseek-r1

ggml_sycl_init: found 3 SYCL devices:

commented

If you do not see the devices listed with clinfo, ensure that you have the permissions to access /dev/dri/renderD*.

sudo gpasswd -a ${USER} render
newgrp render
commented

This is the crux of the problem - this file does not exist. Something sets up a reference to the file, but the filesystem directory structure and file itself are not there: /opt/anaconda1anaconda2anaconda3/lib/libintelocl.so