[Request]: Expose KV Cache Reset functionality.
Nick-infinity opened this issue · comments
I think the kv cache for llm inference is maintained internally by mnn. Is it possible to reset the kv cache manully using an API call?
Marking as stale. No activity in 30 days.