bclarkson-code / Tricycle

Autograd to GPT-2 completely from scratch

Build some optimisers

bclarkson-code opened this issue 7 months ago · comments

bclarkson-code commented 7 months ago

Two optimisers should be added:

SGD with momentum
Adam