Can EAGLE be applied to any auto-regressive model?
sam-sepiola opened this issue · comments
How can I use EAGLE with a custom auto-regressive model that's not supported ( yet!) by default?
Yes, EAGLE doesn't depend on the internal structure of LLM. When training EAGLE with a custom auto-regressive mode, it is important to adjust the instruction template accordingly.
In the ea_model.py:from_pretrained(), you used KVLlamaForCausalLM and KVMixtralForCausalLM as the base_model classes.
If we were to use another model, such as qwen, as the base model, how would we change this part of the code
@thyywr759 We have updated the Readme, and you can find the guide here.