pkglimmer / Gradient-descent-R-demo

Implementation of vanilla gradient descent, Newton’s method, damped Newton’s method, conjugate gradient descent method, momentum, Nesterov accelerated gradient, Adagrad, Adadelta/RMSprop, and Adam.

• vanilla gradient descent
• Newton’s method
• damped Newton’s method
• conjugate gradient descent
• momentum
• Nesterov accelerated gradient
• Adagrad
• Adadelta/RMSprop
• Adam

Functions (optimization and visualization) are all implemented in gradientDescent.R . To run the demo, use RStudio to see the visualized graphs and charts, and tweak the parameters.