SafeAILab / EAGLE

[ICML'24] EAGLE: Speculative Sampling Requires Rethinking Feature Uncertainty

https://arxiv.org/abs/2401.15077

SafeAILab/EAGLE Issues

Llama 3 support?
Updated 2 months ago
new model
Updated 2 months ago3
About reproducing speedup ratio
Updated 2 months ago1
Qs on self.act
Closed 2 months ago1
training efficiency
Updated 2 months ago1
Is there scripts to calculate the overall acceptance rate?
Updated 2 months ago1
About project structure
Closed 2 months ago1
bsne1 branch "last_hidden = out_hidden[ab,last_nopadding][:,None]" gets wrong
Closed 2 months ago4
Inference on the CPU, the speed improvement is limited
Closed 2 months ago2
How many decoder layer needs to add in EAGLE?
Closed 2 months ago5
How to support new model?
Closed 2 months ago1
Can EAGLE be applied to any auto-regressive model?
Closed 2 months ago3
vicuna 7b oom
Closed 4 months ago
Questions about pre-allocated kv_cache.
Closed 2 months ago1
Questions about the EAGLE parameters, not match the pytorch_model.bin
Closed 2 months ago2
About tree_buffer
Closed 3 months ago2
About KVCache of Eagle vs Origin LLM
Closed 3 months ago
run with vllm
Closed 3 months ago1
Question on draft process
Closed 3 months ago1
RuntimeError: probability tensor contains either `inf`, `nan` or element < 0
Closed 4 months ago5
How to train LLama-70B eagle?
Closed 3 months ago4
Generation quality loss
Closed 3 months ago6
Can Eagle improve the inference throughput in continuous batch mode?
Closed 3 months ago4
VLLM contribution
Closed 3 months ago2
can you share ShareGPT dataset used to train the models?
Closed 3 months ago3
KV Cache initialization throwing an error
Closed 4 months ago8
Data processing script (ge_data/allocation.py) script does not work out of the box
Updated 4 months ago1
Support vLLM
Updated 4 months ago
Is there scripts to calculate the acceptance rate at each position?
Closed 4 months ago1
Release Mixtral Training Code
Closed 4 months ago3
Question about manual datasets & Some interests about details
Closed 4 months ago1
7B config
Closed 4 months ago3
runtime
Closed 4 months ago9
Can EAGLE actually improve throughput?
Closed 4 months ago1
Head accuracy
Updated 5 months ago
A simple question regarding to the paper.
Closed 5 months ago1
No pytorch.bin is generated after training is done
Closed 5 months ago2
Eagle head accuracy
Closed 5 months ago6
About reproduction baseline results
Closed 5 months ago2
failed to reproduce given speedup results
Closed 5 months ago1
Small Typo in Blog
Closed 5 months ago1
CUDA out of memory
Closed 5 months ago1
Is the rejection and adjusting probability implementation different from normal speculative sampling?
Closed 5 months ago1
Weired Runtime Error during Inference
Closed 5 months ago2
OSError when download weights from huggingface
Closed 6 months ago3
train configuration
Closed 6 months ago1
Support Chinese Task
Closed 6 months ago1
Support Chinese-LLaMA-Alpaca-2?
Closed 6 months ago3
RuntimeError: indices should be either on cpu or on the same device as the indexed tensor (cuda:7)
Closed 6 months ago2
RuntimeError (porting inference on kaggle)
Closed 6 months ago1