Gradient descent with One-Step Escaping (GOSE)
This repository contains pytorch code that produces the one step escape method (negative curvature descent step) in the paper: Saving Gradient and Negative Curvature Computations: Finding Local Minima More Efficiently. We adopt the one step escape method together with Adam (Adam: A Method for Stochastic Optimization) for training deep networks.
Reference
-
Diederik P. Kingma and Jimmy Ba, Adam: A Method for Stochastic Optimization, 2014
-
Yaodong Yu*, Difan Zou* and Quanquan Gu, Saving Gradient and Negative Curvature Computations: Finding Local Minima More Efficiently, 2017