Giters

SafeAILab / EAGLE

Official Implementation of EAGLE

Home Page:https://arxiv.org/abs/2401.15077

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

Question about manual datasets & Some interests about details

HackGiter opened this issue · comments

commented

Thanks for your great job. I just have some curiosities about it.

First, would it be too big to store the hidden states directly as .ckpt file? How large it is.
Second, could you provide ablation experiment about the cnet and decoding methods?

commented

It depends on the amount of data used and the hidden_size of the target LLM. Using ShareGPT (about 68k conversations), a model with hidden_size=4096 requires 740GB of disk space. The ablation experiment can be found in our paper.