Mozilla-Ocho / llamafile

Distribute and run LLMs with a single file.

Home Page:https://llamafile.ai

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

Llama 3 chat template

woheller69 opened this issue · comments

When using your chat template from here the output starts with "assistant"
https://huggingface.co/jartine/Meta-Llama-3-8B-Instruct-llamafile#prompting

<|begin_of_text|><|start_header_id|>system<|end_header_id|>
{{prompt}}<|eot_id|>{{history}}<|start_header_id|>{{char}}<|end_header_id|>

It works if I change {{char}} to assistant:

<|begin_of_text|><|start_header_id|>system<|end_header_id|>
{{prompt}}<|eot_id|>{{history}}<|start_header_id|>assistant<|end_header_id|>

Can you specify what goes in Template, and Chat Template.
Also there was a problem with stop tokens in early Llama3 how I am sure that is fixed with llamafile?