mistralai / mistral-inference

Official inference library for Mistral models

Home Page:https://mistral.ai/

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

Q: Why rotary embedding applied only to queries and keys?

pmgautam opened this issue · comments

In the codebase, rotary embeddings are applied only to queries and keys but not to values. Can someone point out to reasons/papers behind this design? Thank you in advance!!!

image