ggerganov / ggml

Tensor library for machine learning

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

Why perform such operations on k and v as shown in the above diagram?

EveningLin opened this issue · comments

image
Why perform such operations on k and v as shown in the above diagram?

In order to copy and persist the current key and values (Kcur and Vcur) to the kv cache.