Official inference library for Mistral models
Home Page:https://mistral.ai/
Geek Repo:Geek Repo
Github PK Tool:Github PK Tool
pmgautam opened this issue 3 months ago · comments
In the codebase, rotary embeddings are applied only to queries and keys but not to values. Can someone point out to reasons/papers behind this design? Thank you in advance!!!