HyperRangerMod do not work
hadaev8 opened this issue · comments
/content/DemonRangerOptimizer/optimizers.py in step(self, display, activate_IA, closure)
883 denom = vt.pow_(group['p']).add_(group['eps'])
884
--> 885 n = state['lr']/denom
886
887 if beta3 > 0.0: # apply AdaMod
KeyError: 'lr'
I added state['lr'] = group['lr']
after line 814, since it is done in class HyperRanger
.
Yes, its just a typo
Thanks for pointing out. Updated it.