OverLordGoldDragon / keras-adamw

Keras/TF implementation of AdamW, SGDW, NadamW, Warm Restarts, and Learning Rate multipliers

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

Last weight is updated prematurely (tf.keras)

OverLordGoldDragon opened this issue · comments

Reproducing code -- fix. Only the very last weight in the model is affected. I'll fix this soon.

The effect of this should be minimal in practice; all it does is apply eta_t one t_cur ahead, for the very last weight.