A header-only LBFGS unconstrained optimizer.
LBFGS-Lite is a C/C++ header-only library for unconstrained optimization on twice continuously differentiable (C2) functions or nonsmooth (C0 but piecewise C2) functions. The code is modified from liblbfgs. Only necessary part is preserved for simplicity. Many engineering considerations are also added to improve its robustness over the initial version by Nocedal.
-
Only one header file "lbfgs.hpp" is required for usage.
-
No dependencies except C/C++ standard library.
-
The library is an implementation of Limited-Memory Broyden-Fletcher-Goldfarb-Shanno (LBFGS) with weak or strong Wolfe conditions for smooth functions and the weak Wolfe condition for nonsmooth functions.
-
The objective function is required to be at least C0 but piecewise C2 on its domain.
-
The library provides an additional callback to utilize externally provided maximum feasible stepsize. It can be helpful when the function is closed on a bounded open domain instead of the whole Euclidean space. The callback avoids function evaluations at infeasible region. This can help a lot for closed functions as long as the Newton step is not always clipped.
-
Supports for nonsmooth functions as proposed by Lewis and Overton with a minor modification.
-
Engineering features such as skipping update at extremely small curvature and lower safeguarding for ill-conditioned cases are also adopted.
-
Instruction set dependent parts and L1 regularization parts in original code are removed. Multiple files are reorgainzed here with some additional modification.
See "lbfgs_sample.cpp" for the procedure. See comments in "lbfgs.hpp" for detailed explainations.
Note: If your objective function is really bad (ill-conditioned C0), just set line_search_type = 0 and pass the correct subgradient.
-
LBFGS is only proved to be globally convergent (convergent to stationaries for any initial guess) under convexity assumption, while it also works in many nonconvex cases and yields almost the best results. Slight modification in updating can ensure global convergence without convexity assumption, following a work by Fukushima and Li.
-
Although Lewis-Overton line search and More-Thuente line search are already good enough, we plan to compare them with Hager-Zhang line search and Nocedal's zoom line search. Hager-Zhang linear search is reported to be full machine accuracy while More-Thuente's method can only attain the half. Nocedal's zoom is also reported to be reliable. We will compare them and choose the best one if the improvement is good enough.
-
Further code optimization (instruction set independent) and reorganization.
LBFGS-Lite is distributed under the term of the MIT license according the previous version by Okazaki and the initial FORTRAN version by Nocedal. Please refer to LICENSE file in the distribution.
If any bug, please contact Zhepei Wang (wangzhepei@live.com).