Examples for Llama model architecture
okpatil4u opened this issue · comments
Hello Eric, this looks like great work ! Thank you !!
Can you please add examples for both training and inference for Llama model using candle-lora ? Is it supported through this work ?
Yes! With my candle-lora-macro library, all you need to do is derive the AutoLoraConvert and add the replace_layer_fields to all model structs of a LLama model. They will replace the concrete types and automate the conversion process. Then, call the conversion method on each model struct.
I plan on adding an example shortly. If you have any questions, let me know!
Yes, once you convert to a LoRA model you could fine-tune it. After finetuning, you can merge the weights to speed up inference.
I am closing this so that it does not become a stale issue, but feel free to reopen. I will be adding a LoRA example for Llama and soon!