AttributeError: 'str' object has no attribute 'apply_chat_template'
alew3 opened this issue · comments
I'm trying to get llama-3 working , but when I type anything .. it loads the model and gives this error:
AttributeError: 'str' object has no attribute 'apply_chat_template'
2024-04-19 14:41:24 - 'str' object has no attribute 'apply_chat_template'
Traceback (most recent call last):
File "/Users/username/anaconda3/envs/mlx/lib/python3.11/site-packages/chainlit/utils.py", line 39, in wrapper
return await user_function(**params_values)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/Users/username/projects/ask_azion/SiLLM/app/app.py", line 133, in on_message
prompt = conversation.add_user(message.content)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/Users/username/anaconda3/envs/mlx/lib/python3.11/site-packages/sillm/core/conversation.py", line 81, in add_user
"""
File "/Users/username/anaconda3/envs/mlx/lib/python3.11/site-packages/sillm/core/conversation.py", line 67, in add_message
File "/Users/username/anaconda3/envs/mlx/lib/python3.11/site-packages/sillm/core/conversation.py", line 23, in apply_chat_template
return self.template.apply_chat_template(messages=self.messages, add_generation_prompt=add_generation_prompt)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
AttributeError: 'str' object has no attribute 'apply_chat_template'
I'm using anaconda for the python environment.
Are you running the latest version 0.1.2 with an updated app.py from the repository? This is an error that was fixed since Llama-3 came out.
Try upgrading with pip install sillm-mlx --upgrade and also update the repo using git pull origin.
I just checked with the latest source & build and Llama-3 8B definitely works.
I tried upgrading , and starting from scratch with a new environment .. still getting the error. Not sure what it could be.
my steps to reproduce:
$ conda create -n "sillm" python=3.11
$ conda activate sillm
$ git clone https://github.com/armbues/SiLLM.git
$ cd SiLLM/app
$ pip install sillm-mlx
$ pip install -r requirements.txt
created .env file and setting the SILLM_MODEL_DIR environment variable within the /app folder
$ python -m chainlit run app.py -w
selected the model "llama-3-8b-4bit" and chat template "llama3"
and I still get the error: AttributeError: 'str' object has no attribute 'apply_chat_template'
Looks like I had not pushed the fixed code yet. Sorry for the oversight!
Can you pull the latest app.py and try again?
Yep, working now! Thanks!