Giters

intel / ipex-llm

Accelerate local LLM inference and finetuning (LLaMA, Mistral, ChatGLM, Qwen, DeepSeek, Mixtral, Gemma, Phi, MiniCPM, Qwen-VL, MiniCPM-V, etc.) on Intel XPU (e.g., local PC with iGPU and NPU, discrete GPU such as Arc, Flex and Max); seamlessly integrate with llama.cpp, Ollama, HuggingFace, LangChain, LlamaIndex, vLLM, DeepSpeed, Axolotl, etc.

Repository from Github https://github.comintel/ipex-llmRepository from Github https://github.comintel/ipex-llm

unable to run llama.cpp on 2 A770 cards with x99 platform

luningxie opened this issue · comments

commented

when i run llama.cpp portable version on my ubuntu desktop, there is the wrong message "./llama-cli: the 2 line： 33168 illegal instrutions（Segmentation fault ） LD_LIBRARY_PATH=$LD_LIBRARY_PATH:$(cd "$(dirname "$0")";pwd) $(cd "$(dirname "$0")";pwd)/llama-cli-bin "$@"

my computer config: X99 platform, two way E5-2170 v3 CPUs, 2 A770 video cards, Ubuntu 22.04.05 desktop.

How to reproduce
Steps to reproduce the error:

  1. download llama.cpp portable version, and unzip to the current directory.

  2. open the terminal at the directory and run the following command:
    ./llama-cli -m /llama-cpp-ipex-llm-2.2.0b20250313-ubuntu-xeon/QwQ-32B-Q3_K_L.gguf -p "A conversation between User and Assistant. The user asks a question, and the Assistant solves it. The assistant first thinks about the reasoning process in the mind and then provides the user with the answer. The reasoning process and answer are enclosed within and tags, respectively, i.e., reasoning process here answer here . User: Question:The product of the ages of three teenagers is 4590. How old is the oldest? a. 18 b. 19 c. 15 d. 17 Assistant: " -n 2048 -t 8 -e -ngl 99 --color -c 2500 --temp 0 -no-cnv

3.after running, the screen displays:
ggml_sycl_init: GGML_SYCL_FORCE_MMQ: no
ggml_sycl_init: SYCL_USE_XMX: yes
ggml_sycl_init: found 2 SYCL devices:
/llama-cli: the 2 line： 33168 illegal instrutions（Segmentation fault ） LD_LIBRARY_PATH=$LD_LIBRARY_PATH:$(cd "$(dirname "$0")";pwd) $(cd "$(dirname "$0")";pwd)/llama-cli-bin "$@"

  1. the command stopped and return.

thanks for the help!

commented

llama-cpp-ipex-llm-2.2.0b20250313-ubuntu-xeon is for SPR，E5 v3 is too old. You can try our nightly llama.cpp https://github.com/intel/ipex-llm/blob/main/docs/mddocs/Quickstart/llama_cpp_quickstart.md or our next portable zip(on the way).