mybigday / llama.rn

React Native binding of llama.cpp

Prompt cache

jhen0409 opened this issue 10 months ago · comments

Jhen-Jie Hong commented 10 months ago

https://github.com/ggerganov/llama.cpp/blob/92d0b751a77a089e650983e9f1564ef4d31b32b9/examples/main/main.cpp#L243

Support save prompt cache as file so we could speed up context initialization + prompt processing.