Incompatibility with CuArrays

[Christopher Rackauckas]

MWE:

using NLsolve, CuArrays

function f!(F, x)
    mul!(F,A,x)
    F .-= R
end

A = cu(rand(2,2))
R = cu(rand(2))
nlsolve(f!, cu([ 0.1; 1.2]))

[Charles Kawczynski]

Is there any plan on making this package GPU friendly?

[Patrick Kofod Mogensen]

I'm getting very close to a release of NLSolvers.jl ... 🤷‍♂️
(minimizing the 2-norm of the residual... haven't fixed printing to reflect it was actually an nsolve! call)

julia> x_vec = cu(rand(2))
2-element CuArray{Float32,1,Nothing}:
 0.8914334
 0.7412733

julia> nlsolve!(OnceDiffed(f!), x_vec, TrustRegion(Newton()))
Results of minimizaion

* Algorithm:
  Newton's method with default linsolve with Trust Region (Newton, cholesky)

* Candidate solution:
  Final objective value:    5.55e-16
  Final gradient norm:      1.72e-09

  Initial objective value:  5.35e-02
  Initial gradient norm:    1.13e-01

* Convergence measures
  |x - x'|              = 4.07e-06 <= 0.00e+00 (false)
  |x - x'|/|x|          = 1.29e-07 <= 0.00e+00 (false)
  |f(x) - f(x')|        = 4.79e-14 <= 0.00e+00 (false)
  |f(x) - f(x')|/|f(x)| = 9.89e-01 <= 0.00e+00 (false)
  |g(x)|                = 1.72e-09 <= 1.00e-08 (true)
  |g(x)|/|g(x₀)|        = 1.51e-08 <= 0.00e+00 (false)
  Δ                     = 7.81e+02 <= 0.00e+00 (false)

* Work counters
  Seconds run:   6.62e-03
  Iterations:    4


julia> x_vec
2-element CuArray{Float32,1,Nothing}:
 30.826872
 -7.21932

julia> A\R
2-element CuArray{Float32,1,Nothing}:
 30.826878
 -7.2193217

[Patrick Kofod Mogensen]

(btw, sorry I didn't see this earlier, my notifications for this whole Organization are weird)

[Patrick Kofod Mogensen]

Trust region can be used without autoscaling now. If someone can think of a good way to apply norm in the columns of a matrix on a gpu I'll fix autoscaling

[Charles Kawczynski]

We may be a bit closer, using autodiff = :forward, with ForwardDiff.jl's 472.

[Patrick Kofod Mogensen]

How so?

[Charles Kawczynski]

This operation was previously not GPU-capable