microsoft / mup

maximal update parametrization (µP)

Home Page:https://arxiv.org/abs/2203.03466

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

Interpreting jitter in coordcheck

leenachennuru opened this issue · comments

Hi,

I have coord check plots with mup that look roughly stable but are a bit more jittery compared to the example plots. I was wondering if this can be considered expected behavior.

mup_test

Thanks!

It's good as long as the magnitude is stable. One way to verify further is to extend the x-axis to the right

Thanks! I could verify the stability of LR with width even with this jitter. Closing this issue accordingly.