Mixed partial derivatives

[Tommaso Bianucci]

Is there a way to compute higher order mixed partial derivatives with the current state of the library?
If not, is there a plan to support them in the (near) future?

Thanks! :)

[jacob-m-wilson-42]

Not sure if this is the answer you are looking for, but TaylorDiff does have a directional derivative function. While it does not explicitly evaluate a single mixed partial derivatives, you may be able to formulate it to evaluate what you are looking for. I tried real quick to compose a mixed partial derivative using the below code

func(x,y) = 2 * x^2 * y^2
derivative(  outertemp ->  derivative(temp -> func(temp[1],temp[2]), outertemp ,[1f0,0f0],1), [1f0,1f0],[0f0,1f0],1 )

but it looks like TaylorDiff does not currently support taking derivatives of derivatives. I guess it's kinda by design, since the idea is to use the speed of TaylorDiff at directly evaluating higher order derivatives. If you're unfamiliar with the line of code I wrote here's a quick explanation:

The derivative() function has inputs derivative(func,input,direction,order). I created an anonymous function (for the inner derivative) called temp that takes the vector input and passes the values to the correct locations of the function (probably more efficient to just have a function that directly takes a vector as an input). The direction is the direction of the directional derivative (note that a second order directional derivative is not necessarily the same thing as a standard second order derivative) . Order is the order of the derivative. I've used the syntax 0f0 and 1f0 to initialize the values as floating point 32 bit values (TaylorDiff is a little picky with the data types, it likes the input and direction to have the same type). I wrapped the inner derivative with the outer derivative and made another anonymous function to pass the data to the input section of the inner derivative.

This code does error, but this same syntax should work with other packages such as ForwardDiff. If you are just trying to evaluate derivatives for something, ForwardDiff probably has the most support to do what you are looking for. If you are looking to take these derivatives for use in a neural net, TaylorDiff is the only package I've been able to get to play nicely with the Zygote package that Flux uses.

[Songchen Tan]

The goal is to be able to calculate mixed derivatives like

$$\frac{\mathrm d^5u}{\mathrm dx^2dy^3}$$

as two sequential operation,

# v = [x, y]
u(v) = ...
uxx(v) = derivative(u, v, [1., 0.], 2)
uxxyyy(v) = derivative(uxx, v, [0., 1.], 3)

So that this will be more efficient than nesting first order derivative for five times.

There are currently some code generation problems that prevents me to do that, but I will fix that soon.

[Tommaso Bianucci]

Thanks a lot both for your inputs.
What @tansongchen mentioned is actually what I also had in mind, given also what I understood about the design of TaylorDiff.jl
Thanks for working on it and I am looking forward to test it when ready :)

[Songchen Tan]

Fixed in #59. @churli Please try out and let me know if there are any questions!