harryliew / SdLBFGS

SdLBFGS

This project implements Stochastic damped LBFGS (SdLBFGS) [1] in PyTorch.

Usage:

1. Put sdlbfgs.py and sdlbfgs0.py in YOUR_PYTHON_PATH/site-packages/torch/optim.
   
2. Open YOUR_PYTHON_PATH/site-packages/torch/optim/__init__.py add the following code:

  from .sdlbfgs import SdLBFGS
  from .sdlbfgs0 import SdLBFGS0
  del sdlbfgs
  del sdlbfgs0

3. Save __init__.py and restart your python.
   
4. Just use SdLBFGS as a normal optimizer in PyTorch.
   Some results are presented in [2]. For any problem, please contact Huidong Liu at [e-mail]
   

References:

[1] Xiao Wang, Shiqian Ma, Donald Goldfarb, Wei Liu. "Stochastic quasi-Newton methods for nonconvex stochastic optimization." SIAM Journal on Optimization 27.2 (2017): 927-956.
[2] Yingkai Li and Huidong Liu. "Implementation of Stochastic Quasi-Newton's Method in PyTorch." arXiv preprint arXiv:1805.02338, 2018.