epicure / llama2.js

Inference Llama 2 in one file of pure JavaScript(HTML)

llama2.js

A pure JavaScript port of Karpathy's llama2.c with a simple UI.

How to run

1. Download Karpathy's Llama2 (Orig instructions) parameters pretrained on TinyStories dataset

   wget https://huggingface.co/karpathy/tinyllamas/resolve/main/stories15M.bin
   wget https://huggingface.co/karpathy/tinyllamas/resolve/main/stories42M.bin
   wget https://huggingface.co/karpathy/tinyllamas/resolve/main/stories110M.bin

2. Open run.html via a WebServer

    python -m http.server 8080
    open http://localhost:8080/run.html

Performance

Tokens/sec measurement on Apple M1

tok/s	15M	42M	110M
🐢	~30	~13	~5

License

MIT