Update llama 3 prompt template
tudoanh opened this issue · comments
Hi, I'm trying to run Llama 3 8B Q4 model, but seems like the prompt template has been changed.
Then I saw this new release from llama.cpp: https://github.com/ggerganov/llama.cpp/releases/tag/b2707
llama : add llama-3 chat template (#6751)
* Added llama-3 chat template
* Update llama.cpp
Can you update so we can run Llama 3 normally? Thanks a lot!
@tudoanh how are you running Llama 3? I'd love to know steps to run for when this issue is addressed.
I'm just using the Llamafile latest ver 0.7.3 with command like this
sh -c './llamafile-0.7.3 -m Meta-Llama-3-8B-Instruct.Q4_0.gguf -ngl 15 --port 8080 --host 0.0.0.0'
You can find the gguf files on huggingface. Example: https://huggingface.co/bartowski/Meta-Llama-3-8B-Instruct-GGUF