Lightning-AI / lit-llama

Implementation of the LLaMA language model based on nanoGPT. Supports flash attention, Int8 and GPTQ 4bit quantization, LoRA and LLaMA-Adapter fine-tuning, pre-training. Apache 2.0-licensed.

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

How to convert hf weight of 70b to lit-lamma weights?

sfarzi opened this issue · comments

I would like to fine-tune (adapt v2) of 70B parameters.
when I tried to convert weights, I came across by an error.
Error
File "/scripts/convert_hf_checkpoint.py", line 42, in convert_hf_checkpoint
config = LLaMAConfig.from_name(model_size)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/lit-llama/lit_llama/model.py", line 38, in from_name
return cls(**llama_configs[name])
~~~~~~~~~~~~~^^^^^^
KeyError: '70B'

the command :

python /scripts/convert_hf_checkpoint.py --output_dir /lit-llama/checkpoints/lit-llama/70B --model_size 70B --checkpoint_dir Llama-2-70b-hf

Any help would be appreciated.