missing Jacobian and Hessian methods

[Carlo Lucibello]

eventually trough Jacobian-vector product. Nice related blog post https://j-towns.github.io/2017/06/12/A-new-trick.html

[jbrea]

I see in @CarloLucibello's PR #57 how to compute hessians with the old interface.
Is there an easy way to compute hessians also with the new interface?

[denizyuret]

You can still use the old interface, here is an example from our SGDDynamics project:

function hessian(loss,w,x,y)
    ∇loss = grad(loss)
    ∇lossi(w,x,y,i) = ∇loss(w,x,y)[i]
    ∇∇lossi = grad(∇lossi)
    w = value(w)
    n = length(w)
    h = similar(Array(w),n,n)
    for i in progress(1:n)
        h[:,i] .= Array(vec(∇∇lossi(w,x,y,i)))
    end
    return h
end