Which optimizer should I use?

[DKGH]

I have a pair of vectors: x and y. x is just a linspace thing. y is close to a differentiable function f(x). To fit my function, I need to minimize (y - f(x))^2. y is not differentiable. Which method should I use here?

I have tried LBFGS, which gives good results (but it may still be trapped in a local minima). I have also tried simulated annealing. However that gives arbitrary results.

[Marcus Edel]

Have you tried, PSO?

[DKGH]

This is working better. Thanks.