[Doc]: Doc for using tensorizer_uri with LLM is incorrect
GRcharles opened this issue Β· comments
Charles Fauman commented
π The doc issue
This code snippiet doesn't work - tensorizer_uri doesn't exist as a parameter
llm = LLM(model="facebook/opt-125m",
load_format="tensorizer",
tensorizer_uri=path_to_opt_tensors,
num_readers=3,
vllm_tensorized=True)
vllm/examples/tensorize_vllm_model.py
Line 91 in 702bee4
Suggest a potential alternative/fix
No response
Vinay R Damodaran commented
Relevant #4752 (comment)