junyuan-chen / NonlinearSystems.jl

Solve nonlinear systems of equations and nonlinear least squares in Julia

NonlinearSystems.jl

Solve nonlinear systems of equations and nonlinear least squares in Julia

NonlinearSystems.jl is a Julia package for solving nonlinear systems of equations and nonlinear least squares. It renovates well-trusted solution algorithms with highly performant and extensible implementation in native Julia language.

NonlinearSystems.jl places special emphasis on

• Low number of evaluations needed for updating the Jacobian matrix
• Flexibility of swapping the underlying linear solver based on array type and hardware
• Zero memory allocation incurred in iteration steps

At this moment, the only solution algorithm implemented is a modified version of Powell's hybrid method (a trust region method with dogleg). Relations to existing packages are further discussed towards the end.

Quick Start

NonlinearSystems.jl adopts the CommonSolve.jl interface and wraps a residual function as OnceDifferentiable defined in NLSolversBase.jl with an optionally user-provided Jacobian function. The same interface is shared for solving a root-finding problems and solving a least-squares problems.

using NonlinearSystems

# Residual function
function f!(F, x)
    F[1] = (x[1] + 3) * (x[2]^3 - 7) + 18
    F[2] = sin(x[2] * exp(x[1]) - 1)
    return F
end

# Jacobian function (optional)
function j!(J, x)
    J[1,1] = x[2]^3 - 7
    J[1,2] = 3 * x[2]^2 * (x[1] + 3)
    u = exp(x[1]) * cos(x[2] * exp(x[1]) - 1)
    J[2,1] = x[2] * u
    J[2,2] = u
    return J
end

# Initial value
x0 = [0.1, 1.2]

# Evaluate Jacobians via finite differencing methods from FiniteDiff.jl
solve(Hybrid{RootFinding}, f!, x0)

# Use user-specified Jacobian function and separate out the initialization step
s = init(Hybrid{LeastSquares}, f!, j!, x0)
solve!(s)

For more details, please see the documentation.

Related Packages

NonlinearSystems.jl addresses the following limitations that the related packages do not:

• MINPACK.jl and GSL.jl
  • No option for swapping the linear solver
  • Use of rank-1 update of the Jacobian matrix and factorization cannot be adjusted
  • Wrappers of C interface; no native Julia implementation
  • MINPACK.jl does not work on Apple Silicon
• NLsolve.jl
  • No option for swapping the linear solver
  • No rank-1 update of the Jacobian matrix and factorization
  • Iteration steps are not non-allocating
  • No support for nonlinear least squares
• NonlinearSolve.jl
  • Trust region solver (TrustRegion) does not provide correct answers on test problems (as of version 1.6)
  • No rank-1 update of the Jacobian matrix and factorization
  • Iteration steps are not non-allocating
  • No support for nonlinear least squares
• LeastSquaresOptim.jl
  • Only solves nonlinear least squares
  • No rank-1 update of the Jacobian matrix and factorization
• LsqFit.jl
  • Only solves nonlinear least squares
  • Performance seems to be dominated by LeastSquaresOptim.jl

Roadmap

The development of NonlinearSystems.jl is still in an early stage. At this moment, only trust-region methods are considered and the linear problem involved in each iteration is only solved by dense matrix factorization.

The following features will be added in future:

• Support for sparse Jacobian matrices
• Support for conducting linear algebra on GPUs

References

Moré, Jorge J., Danny C. Sorenson, Burton S. Garbow, and Kenneth E. Hillstrom. 1984. "The MINPACK Project." In Sources and Development of Mathematical Software, ed. Wayne R. Cowell, 88-111. New Jersey: Prentice-Hall.

Nocedal, Jorge, and Stephen J. Wright. 2006. Numerical Optimization. 2nd ed. New York: Springer.

Powell, Michael J. D. 1970. "A Hybrid Method for Nonlinear Equations." In Numerical Methods for Nonlinear Algebraic Equations, ed. Philip Rabinowitz, 87-114. London: Gordon and Breach.