hpcaitech / SwiftInfer

Efficient AI Inference & Serving

Home Page:https://hpc-ai.com/

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

cannot import name 'LoraConfig' from 'tensorrt_llm.runtime.lora_manager'

chaos318 opened this issue · comments

It seems version manager is corrupted。
first:if use pip install tensorrt_llm==0.6.0。you will get “ERROR: Could not find a version that satisfies the requirement tensorrt_llm==0.6.0 (from versions: 0.5.0, 0.5.0.post1, 0.6.1, 0.7.0, 0.7.1)”
second:if install and use 0.6.1 you will get “ImportError: cannot import name 'LoraConfig' from 'tensorrt_llm.runtime.lora_manager' (/usr/local/lib/python3.10/dist-packages/tensorrt_llm/runtime/lora_manager.py)”。because LoraConfig first find in v0.7.0。
third: if install 0.7.0 . you will get another error : "AttributeError: 'LLaMAForCausalLM' object has no attribute 'get_transformer_layers'" because SwiftInfer package have used get_transformer_layers。
It can't work anyhow。