How to convert hf weight of 70b to lit-lamma weights?
sfarzi opened this issue · comments
I would like to fine-tune (adapt v2) of 70B parameters.
when I tried to convert weights, I came across by an error.
Error
File "/scripts/convert_hf_checkpoint.py", line 42, in convert_hf_checkpoint
config = LLaMAConfig.from_name(model_size)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/lit-llama/lit_llama/model.py", line 38, in from_name
return cls(**llama_configs[name])
~~~~~~~~~~~~~^^^^^^
KeyError: '70B'
the command :
python /scripts/convert_hf_checkpoint.py --output_dir /lit-llama/checkpoints/lit-llama/70B --model_size 70B --checkpoint_dir Llama-2-70b-hf
Any help would be appreciated.