Weird behavior with finite-differencing

[Misha Mikhasenko]

I was looking for a simple solver for a single non-linear equation, NLsolve seems to be what I need.
But, often it throws NaN even with the innocent example x->(x^2+t).

g(t) = nlsolve((f, x)->f[1]=abs2((x[1]+1im*x[2])^2+t), [3.1,0.1])
for i in 1:5
    @show complex(g(-4.0).zero...)
end

output:

complex(g(-4.0).zero...) = NaN + NaN*im
complex(g(-4.0).zero...) = NaN + NaN*im
complex(g(-4.0).zero...) = 2.000006535131516 + 8.175647538490073e-7im
complex(g(-4.0).zero...) = 2.155735781016897 + 0.4355915357896875im
complex(g(-4.0).zero...) = NaN + NaN*im

It seems that NaN appears when optimizer starts moving in a wrong direction, which obviously happens based on the calculated derivative. I can not use autodiff=:forward because it has the problem with Dual numbers. But I checked that when the derivative provided, the result is fine.

g(t) = nlsolve((f, x)->f[1]=(x[1]^2-x[2]^2+t)^2+(2*x[1]*x[2])^2, [3.1,0.1], autodiff=:forward)
for i in 1:5
    @show complex(g(-4.0).zero...)
end

complex(g(-4.0).zero...) = 2.0000065350959733 + 8.175557204820513e-7im
complex(g(-4.0).zero...) = 2.0000065350959733 + 8.175557204820513e-7im
complex(g(-4.0).zero...) = 2.0000065350959733 + 8.175557204820513e-7im
complex(g(-4.0).zero...) = 2.0000065350959733 + 8.175557204820513e-7im
complex(g(-4.0).zero...) = 2.0000065350959733 + 8.175557204820513e-7im```

[Patrick Kofod Mogensen]

Interesting, thanks for reporting that. Could you show the relevant traces?

[Misha Mikhasenko]

For the traces:

function g(t)
    function f!(f,x)
        f[1] = abs2((x[1]+1im*x[2])^2+t)
        println("x = $x, f = $f")
    end
    nlsolve(f!, [3.1,0.1])
end
@show complex(g(-4.0).zero...)

x = [3.1, 0.1], f = [31.7444, 0.1]
x = [3.10002, 0.1], f = [31.7457, 0.0]
x = [3.09998, 0.1], f = [31.7431, -3.41322e-304]
x = [3.1, 0.100006], f = [31.7444, 0.0]
x = [3.1, 0.0999939], f = [31.7444, -3.41322e-304]
x = [NaN, NaN], f = [NaN, 0.1]
x = [NaN, NaN], f = [NaN, 0.1]
x = [NaN, NaN], f = [NaN, 0.1]
x = [NaN, NaN], f = [NaN, 0.1]
x = [NaN, NaN], f = [NaN, 0.1]
x = [NaN, NaN], f = [NaN, 0.1]

why f is two dimensional? Can I specify somewhere that my x is two-dimensional, my f is one-dimensional?

[Spencer Lyon]

I may be remembering wrong, but I think that this package only solves for a solution to N equations in N variables.

If you have a scalar objective function, perhaps you want Optim.jl?

[Patrick Kofod Mogensen]

Yup, the game is from N to N. What happens is that sometimes f[2] is a very small number and everything goes well, but if it's not, then everything can happen. You should indeed try Optim.jl (and there you can just pass in the complex number directly)

[Patrick Kofod Mogensen]

Soo, this is fixed by using an appropriate method from another package, so I'm closing this.

[Misha Mikhasenko]

Thank you for considering the issue. Indeed I solved my problem with Optim.jl.
Concerning NaN from in the finite-differencing, I would think it still should not happen, i.e. there is no reason for the error to happen.

Even if the package is not applicable here, I can imagine that there are cases NxN where the same problem arises. I can try to find such example if it will help you guys to polish the package.

[Patrick Kofod Mogensen]

I’m pretty sure this only happens because you’re not updating all the elements as is assumed by the package, but if I’m wrong I’d be happy to fix it!