openai / grok

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

questions regarding modulus division

garyhlai opened this issue · comments

Thanks for the very interesting paper! I have two questions regarding modular division:

  1. Does figure 1 of the paper correspond to this equation x◦y=x/y (mod p) for 0≤x<p, 0<y<p where p == 97?
  2. Wouldn't x/y (mod p) produce fractional results? How would you get the cross-entropy loss (against these fractional targets) then?

I tried staring at the code but couldn't really connect the dots :(

97 is a prime number, and the integers modulo a prime power form a field. Specifically, that means multiplication forms an Abelian group, so division is well defined for all elements (except zero) and closed over the field.

i.e. division of integers modulo 97 will produce another integer (module 97), since every element has a multiplicative inverse.