Support Chinese-LLaMA-Alpaca-2?

Question

Support Chinese-LLaMA-Alpaca-2?

skykiseki opened this issue 6 months ago · comments

skykiseki commented 6 months ago

Nice work!
I just wonder will EAGLE support Chinese-LLaMA-Alpaca-2 and will you show the paper in the future? ?

model url: https://github.com/ymcui/Chinese-LLaMA-Alpaca-2

Hope for your response, thx

skykiseki · Answer 1 · Wed Dec 20 2023 20:21:42 GMT+0800 (China Standard Time)

And I want to know the difference between EAGLE and SpecInfer. I'm guessing that the tree verification part of the two is the same, right?

yuhuili · Answer 2 · Thu Dec 21 2023 10:48:22 GMT+0800 (China Standard Time)

We will support more models in the future.

Yes, the tree verification part is the same. Our main contribution lies in the guessing and drafting part, which is different from SpecInfer. Also, the application scope of EAGLE is different from SpecInfer. EAGLE is not limited to distributed LLM inference or offloading-based LLM inference, and it can be used in more general situations.

skykiseki · Answer 3 · Thu Dec 21 2023 11:33:35 GMT+0800 (China Standard Time)

We will support more models in the future.

Yes, the tree verification part is the same. Our main contribution lies in the guessing and drafting part, which is different from SpecInfer. Also, the application scope of EAGLE is different from SpecInfer. EAGLE is not limited to distributed LLM inference or offloading-based LLM inference, and it can be used in more general situations.

OK,Looking forward to the upcoming updates, thank you.