Prototype of a specialized algorithm for LCP

[Jesse Perla]

@arnavs cc: @pkofod
As discussed, it would be great to check if a specialized algorithm for LCP problems has higher performance than the general nonlinear one.

Yuval has given permission to port over https://www.mathworks.com/matlabcentral/fileexchange/20952-lcp-mcp-solver-newton-based?s_tid=prof_contriblnk

I think we should, before trying to hook it into NLsolve:

• Do a direct port over of the LCP function there, being careful with preallocating vectors/type stability/etc. but otherwise not do any enhancements
• Verify that it gives the same solution as the matlab one
• Compare the performance to PATH's LCP and the MCP in NLsolve. If the performance is sufficiently better than the current NLsolve, then we can take the next steps.

[Antoine Levitt]

Doesn't Jump support this? It looks like a well studied class of problems for which there should be efficient solvers there.

[Arnav Sood]

@antoine-levitt I had trouble using the VariationalInequality and Complementarity packages in JuMP, mainly because of the vectorized map. If you'd like to take a crack at it, here's a minimal setup of the problem in PATHSolver.

using PATHSolver, LinearAlgebra
n = 50
B = Tridiagonal(rand(n-1), rand(n), rand(n-1))
q = rand(n)
f = z -> B*z + q
lb = -1000. * ones(n)
ub = 1000. * ones(n)
options(convergence_tolerance = 1e-12, output = :yes, time_limit = 600) # 10 minute budget
@time exit_code, sol_z, sol_f = solveLCP(f, lb, ub)

[Arnav Sood]

@antoine-levitt Actually, after some inspection, this is not bad at all in pure JuMP. So maybe the takeaway is that the two specialized packages above are ill-adapted for this.

One straightforward implementation is:

using JuMP, Ipopt, LinearAlgebra
# setup objects
n = 50 
tolerance = 1e-12
B = Tridiagonal(rand(n-1), rand(n), rand(n-1))
q = rand(n)
# define and solve model
m = Model(solver = IpoptSolver(tol = tolerance))
@variable(m, z[1:n])
@constraint(m, z .>= 0)
@constraint(m, B*z + q .>= 0)
@objective(m, Min, z' * B * z + z' * q)
solve(m)

[Jesse Perla]

The quadratic programming formulation might work, but there are specialized algorithms that could dominate it... I don't think people use the QP formulation for these in practice (see http://fmwww.bc.edu/ec-p/software/Miranda/chapt4.pdf for a few of the algorithms). Lemke and others are the standards, I believe.

That said, perhaps a commercial quality QP solver would win against a more specialized implementation done by us.

[Arnav Sood]

I’ll benchmark the commercial solvers.

[Patrick Kofod Mogensen]

Stupid question time: is it possible to AD through a JuMP.solve call?

[Patrick Kofod Mogensen]

Had an actual look at the code and paper. In principle all you have to do is to do the FB transform, and then pass it to curve_fit in LsqFit.

[Arnav Sood]

FYI, will maintain a table here with the results on the 1000-node diffusion problem.

Mosek (QP formulation) --- 4.1 e-6 seconds 
PATHSolver  --- 13.6 e-6 seconds 
OSQP (QP formulation) --- 164 e-6 seconds 
Ipopt --- 171 e-6 seconds 
NLsolve --- 589 e-6 seconds

Note that NLsolve scales particularly poorly in the jump from 300 to 1000 nodes (it's a whole new order of magnitude slower).

To Do:

• Eliminate PSD error in Gurobi
• Try other formulations (i.e., direct LCP).
• Give Yuval's thing a shot.

[Jesse Perla]

Might be worth trying the exact matlab thing from Ben's code using Yuvals stuff with 1000 nodes to see how it goes

[Arnav Sood]

Sure, will give it a shot tomorrow. Should give us a sense of the value of porting.

[Patrick Kofod Mogensen]

Could you share the script?

[Arnav Sood]

@jlperla You asked for timing information on the solveMCP coming out of PATHSolve. I ran the code again (during lunch, on my laptop). Here's what we have:

solveLCP (PATHSolve): 3.955 ms
solveMCP (PATHSolve): 14.159 ms
NLSolve: 76.867 ms
JuMP/IPOPT: 99.152 ms
JuMP/OSQP: 45.953 ms

I think with the exception of OSQP, we're in roughly the same tolerance ballpark. Just in case it matters:

uarnavsood@mbp:~$ julia
               _
   _       _ _(_)_     |  Documentation: https://docs.julialang.org
  (_)     | (_) (_)    |
   _ _   _| |_  __ _   |  Type "?" for help, "]?" for Pkg help.
  | | | | | | |/ _` |  |
  | | |_| | | | (_| |  |  Version 1.1.0 (2019-01-21)
 _/ |\__'_|_|_|\__'_|  |  Official https://julialang.org/ release
|__/                   |

julia> versioninfo()
Julia Version 1.1.0
Commit 80516ca202 (2019-01-21 21:24 UTC)
Platform Info:
  OS: macOS (x86_64-apple-darwin14.5.0)
  CPU: Intel(R) Core(TM) i7-8750H CPU @ 2.20GHz
  WORD_SIZE: 64
  LIBM: libopenlibm
  LLVM: libLLVM-6.0.1 (ORCJIT, skylake)