willtownes / optimization-practice

sandbox for practice implementing optimization algorithms

Optimization Practice

Will Townes

This is a sandbox for trying different optimization algorithms on toy problems to build intuition. Some algorithms I plan to test out:

• irprop+ ("improved" rprop)
• gradient descent with momentum
• gradient descent with nesterov momentum