numericalalgorithmsgroup / dfols

Python-based Derivative-Free Optimizer for Least-Squares

DFO-LS: Derivative-Free Optimizer for Least-Squares

DFO-LS is a flexible package for solving nonlinear least-squares minimization, without requiring derivatives of the objective. It is particularly useful when evaluations of the objective function are expensive and/or noisy. DFO-LS is more flexible version of DFO-GN.

This is an implementation of the algorithm from our paper: C. Cartis, J. Fiala, B. Marteau and L. Roberts, Improving the Flexibility and Robustness of Model-Based Derivative-Free Optimization Solvers, ACM Transactions on Mathematical Software, 45:3 (2019), pp. 32:1-32:41 [preprint]. For reproducibility of all figures in this paper, please feel free to contact the authors.

If you are interested in solving general optimization problems (without a least-squares structure), you may wish to try Py-BOBYQA, which has many of the same features as DFO-LS.

Documentation

See manual.pdf or here.

Citation

If you use DFO-LS in a paper, please cite:

Cartis, C., Fiala, J., Marteau, B. and Roberts, L., Improving the Flexibility and Robustness of Model-Based Derivative-Free Optimization Solvers, ACM Transactions on Mathematical Software, 45:3 (2019), pp. 32:1-32:41.

If you use DFO-LS for problems with constraints, including bound constraints, please also cite:

Hough, M. and Roberts, L., Model-Based Derivative-Free Methods for Convex-Constrained Optimization, arXiv preprint arXiv:2111.05443, (2021).

Requirements

DFO-LS requires the following software to be installed:

• Python 2.7 or Python 3 (http://www.python.org/)

Additionally, the following python packages should be installed (these will be installed automatically if using pip, see Installation using pip):

• NumPy 1.11 or higher (http://www.numpy.org/)
• SciPy 1.11 or higher (http://www.scipy.org/)
• Pandas 0.17 or higher (http://pandas.pydata.org/)

Optional package: DFO-LS versions 1.2 and higher also support the trustregion package for fast trust-region subproblem solutions. To install this, make sure you have a Fortran compiler (e.g. gfortran) and NumPy installed, then run pip install trustregion. You do not have to have trustregion installed for DFO-LS to work, and it is not installed by default.

Installation using conda

DFO-LS can be directly installed in Anaconda environments using conda-forge:

$ conda install -c conda-forge dfo-ls

Installation using pip

For easy installation, use pip as root:

$ [sudo] pip install DFO-LS

or alternatively easy_install:

$ [sudo] easy_install DFO-LS

If you do not have root privileges or you want to install DFO-LS for your private use, you can use:

$ pip install --user DFO-LS

which will install DFO-LS in your home directory.

Note that if an older install of DFO-LS is present on your system you can use:

$ [sudo] pip install --upgrade DFO-LS

to upgrade DFO-LS to the latest version.

Manual installation

Alternatively, you can download the source code from Github and unpack as follows:

$ git clone https://github.com/numericalalgorithmsgroup/dfols
$ cd dfols

DFO-LS is written in pure Python and requires no compilation. It can be installed using:

$ [sudo] pip install .

If you do not have root privileges or you want to install DFO-LS for your private use, you can use:

$ pip install --user .

instead.

To upgrade DFO-LS to the latest version, navigate to the top-level directory (i.e. the one containing setup.py) and rerun the installation using pip, as above:

$ git pull
$ [sudo] pip install .  # with admin privileges

Testing

If you installed DFO-LS manually, you can test your installation by running:

$ python setup.py test

Alternatively, the HTML documentation provides some simple examples of how to run DFO-LS.

Examples

Examples of how to run DFO-LS are given in the documentation, and the examples directory in Github.

Uninstallation

If DFO-LS was installed using pip you can uninstall as follows:

$ [sudo] pip uninstall DFO-LS

If DFO-LS was installed manually you have to remove the installed files by hand (located in your python site-packages directory).

Bugs

Please report any bugs using GitHub's issue tracker.

License

This algorithm is released under the GNU GPL license. Please contact NAG for alternative licensing.