Giters

jankais3r / LLaMA_MPS

Run LLaMA (and Stanford-Alpaca) inference on Apple Silicon GPUs.

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

Fine tuning LLaMA on Apple Silicon GPUs

Gincioks opened this issue · comments

commented

Hello,

I am new to the AI field and still trying to understand how things work. I was wondering if it's possible to apply this implementation in the fine-tuning process. Like: https://github.com/lxe/llama-tune or https://github.com/tloen/alpaca-lora

I would be grateful for any examples or tutorials that explain how to apply this implementation to the fine-tuning process. Thank you in advance for your help!

commented

Hi, I experimented with it, but so far I am running out of memory on a 128GB machine. Training and fine tuning requires significantly more memory than inference, so I am not sure I will be able to get the memory usage low enough for Apple Silicon Macs... My hope is that tloen or lxe will eventually release their fine tuned models.

commented

The latest commit contains support for running alpaca inference.