Giters

intel-analytics / ipex-llm

Accelerate local LLM inference and finetuning (LLaMA, Mistral, ChatGLM, Qwen, Baichuan, Mixtral, Gemma, Phi, etc.) on Intel CPU and GPU (e.g., local PC with iGPU, discrete GPU such as Arc, Flex and Max); seamlessly integrate with llama.cpp, Ollama, HuggingFace, LangChain, LlamaIndex, DeepSpeed, vLLM, FastChat, Axolotl, etc.

Home Page:https://ipex-llm.readthedocs.io

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

Could ipex-llm support llama3 or the other llm QLora finetune?

sanbuphy opened this issue · comments

commented

Thank you !

commented

Hi, @sanbuphy
We support llama3 qlora finetuning, and also support llama2, chatglm3 and etc. Please refer to here for detailed example usage.

commented

Hi, @sanbuphy We support llama3 qlora finetuning, and also support llama2, chatglm3 and etc. Please refer to here for detailed example usage.

Thank you very much .

commented

Hi, @sanbuphy We support llama3 qlora finetuning, and also support llama2, chatglm3 and etc. Please refer to here for detailed example usage.

Hi, but it's GPU finetune, Do we have CPU finetune demo like qwen? I have noticed this https://github.com/intel-analytics/ipex-llm/tree/main/python/llm/example/CPU/QLoRA-FineTuning but i think maybe too simple

commented

Hi, @sanbuphy We support llama3 qlora finetuning, and also support llama2, chatglm3 and etc. Please refer to here for detailed example usage.

Hi, but it's GPU finetune, Do we have CPU finetune demo like qwen? I have noticed this https://github.com/intel-analytics/ipex-llm/tree/main/python/llm/example/CPU/QLoRA-FineTuning but i think maybe too simple

We don't have CPU demo for Qwen right now. But, CPU and GPU dependencies are almost the same. GPU examples can be changed to CPU examples with a few changes.

We can add more fine-tune examples/models for CPUs later. :)