Why perform such operations on k and v as shown in the above diagram?
EveningLin opened this issue · comments
EveningLin commented
Yavor Ivanov commented
In order to copy and persist the current key and values (Kcur and Vcur) to the kv cache.
Tensor library for machine learning
EveningLin opened this issue · comments
In order to copy and persist the current key and values (Kcur and Vcur) to the kv cache.