I tweaked some parts of the chat.py file to running. The file is from mlc-llm
Follow these steps to run mlc-llm chat.py
tvm relax
git clone https://github.com/mlc-ai/relax.git --recursive
Then try
import tvm
from tvm import relax
mkdir -p dist
git lfs install
git clone https://huggingface.co/mlc-ai/demo-vicuna-v1-7b-int3 dist/vicuna-v1-7b
git clone https://github.com/mlc-ai/binary-mlc-llm-libs.git dist/lib
Step 3: Copy the dist folder and mlc_llm folder into the project directory.
To run the project, execute the following command:
python chat.py
Then You can chat with LLM or do something with python !