How many decoder layer needs to add in EAGLE?

[yjdy]

Thanks for the great work. I find a different between code and paper

In the paper, it say eagle trains a decoder layer.
In the code, cnet.py class Model (https://github.com/SafeAILab/EAGLE/blob/main/model/cnets.py#L491)

self.layers = nn.ModuleList([LlamaDecoderLayer(config, index) for index in range(config.num_hidden_layers)])

It needs to add a decoder layer for every hidden layer.

Please tell me which one is correct.

Best regards

[Chengyuan Li]

This config is not the config file in your llm directory, check the config.json in your ea_model_path, config.num_hidden_layers should be 1.

[yjdy]

Thanks for the response.
If I need to train eagle on different LLM, I think I should prepare different config file.
I just need to copy the config from the LLM I want to train on and change the num_hidden_layers to 1, right?

[Chengyuan Li]

Not 100% sure, you better check the code in train/main.py

[yuhuili]

If I need to train eagle on different LLM, I think I should prepare different config file. I just need to copy the config from the LLM I want to train on and change the num_hidden_layers to 1, right?

You are right.

[yuhuili]

I find a different between code and paper.

Thank you for your interest. As @cyLi-Tiger mentioned, the configuration of EAGLE is different from the base LLM.