Incompatibility with CuArrays
ChrisRackauckas opened this issue · comments
MWE:
using NLsolve, CuArrays
function f!(F, x)
mul!(F,A,x)
F .-= R
end
A = cu(rand(2,2))
R = cu(rand(2))
nlsolve(f!, cu([ 0.1; 1.2]))
Is there any plan on making this package GPU friendly?
I'm getting very close to a release of NLSolvers.jl ... 🤷♂️
(minimizing the 2-norm of the residual... haven't fixed printing to reflect it was actually an nsolve!
call)
julia> x_vec = cu(rand(2))
2-element CuArray{Float32,1,Nothing}:
0.8914334
0.7412733
julia> nlsolve!(OnceDiffed(f!), x_vec, TrustRegion(Newton()))
Results of minimizaion
* Algorithm:
Newton's method with default linsolve with Trust Region (Newton, cholesky)
* Candidate solution:
Final objective value: 5.55e-16
Final gradient norm: 1.72e-09
Initial objective value: 5.35e-02
Initial gradient norm: 1.13e-01
* Convergence measures
|x - x'| = 4.07e-06 <= 0.00e+00 (false)
|x - x'|/|x| = 1.29e-07 <= 0.00e+00 (false)
|f(x) - f(x')| = 4.79e-14 <= 0.00e+00 (false)
|f(x) - f(x')|/|f(x)| = 9.89e-01 <= 0.00e+00 (false)
|g(x)| = 1.72e-09 <= 1.00e-08 (true)
|g(x)|/|g(x₀)| = 1.51e-08 <= 0.00e+00 (false)
Δ = 7.81e+02 <= 0.00e+00 (false)
* Work counters
Seconds run: 6.62e-03
Iterations: 4
julia> x_vec
2-element CuArray{Float32,1,Nothing}:
30.826872
-7.21932
julia> A\R
2-element CuArray{Float32,1,Nothing}:
30.826878
-7.2193217
(btw, sorry I didn't see this earlier, my notifications for this whole Organization are weird)
Trust region can be used without autoscaling now. If someone can think of a good way to apply norm
in the columns of a matrix on a gpu I'll fix autoscaling
We may be a bit closer, using autodiff = :forward
, with ForwardDiff.jl's 472.
How so?
This operation was previously not GPU-capable