Giters

SafeAILab / EAGLE

Official Implementation of EAGLE-1 and EAGLE-2

Home Page:https://arxiv.org/pdf/2406.16858

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

RuntimeError (porting inference on kaggle)

rkuo2000 opened this issue · comments

commented

RuntimeError: Failed to import transformers.modeling_utils because of the following error (look up to see its traceback):
Failed to import transformers.generation.utils because of the following error (look up to see its traceback):
cannot import name 'builder' from 'google.protobuf.internal' (/opt/conda/lib/python3.10/site-packages/google/protobuf/internal/init.py)

commented

fixed, after comment !pip install -r requirements.txt, and execute
pip install torch transformers accelerate fschat gradio openai anthropic sentencepiece protobuf
also
base_model_path = "lmsys/vicuna-7b-v1.3"
EAGLE_model_path = "yuhuili/EAGLE-Vicuna-7B-v1.3"

For your reference: https://www.kaggle.com/code/rkuo2000/eagle-llm/notebook