Matrices
ChrisRackauckas opened this issue · comments
I am wondering how exactly Reduce.jl is handling matrices. I see that REDUCE itself has matrix support: are you using that or relying on generic algorithms on Julia matrices or calling into REDUCE?
The key that I am looking for is, for J a symbolic Jacobian, calculating (I-gamma*J)^-1 for small enough Jacobians (size <16?) and simplifying the expression. A SymEngine based version in ParameterizedFunctions.jl which works okay but is all pointers to C++ variables which ruins precompilation and is quite different from working on Julia ASTs. ModelingToolkit.jl is using a Julia-defined operation tree which isn't type stable so it's as slow as you'd expect. REDUCE looks like it should have the answer here, so I'm curious to see what it looks like.
This is what I was still in the process of figuring out at the moment. The latest commit on github has some additional matrix support, but it is not optimized yet. In some cases it uses Julia's built-in generic algorithms and in some cases it converts the whole matrix to a single Reduce call and translates it back. Of course, the efficient way to do it is to have a single call to Reduce, instead of relying on the generic algorithm, since then the calculation might involve many calls to Reduce. This is mostly an issue of multiple dispatch and experimenting. I was in the process of trying to optimize this aspect and have some stashed changes that I have not commited yet. In the next few days I have to finalize my school work, but I will be trying to work out the details as soon as that is done.
The parser itself can translate matrices back and forth from Array types and Expr types, it is becoming quite versatile, although there are some details that still need to be refined.
Yes, I noticed that calculating the Jacobian is one of your main concerns. I think we should be able to work something out to get the behavior that is needed and to make it efficient.
Tthe jacobian method has been added to the ReduceLinAlg package, which is dedicated to the LINALG extra package included with Reduce binaries.
julia> using ReduceLinAlg
julia> eqns = [:x1-:x2, :x1+:x2-:x3+:x6t, :x1+:x3t-:x4, 2*:x1tt+:x2tt+:x3tt+:x4t+:x6ttt, 3*:x1tt+2*:x2tt+:x5+0.1*:x8, 2*:x6+:x7, 3*:x6+4*:x7, :x8-sin(:x8)]
8-element Array{Expr,1}:
:(x1 - x2)
:(x1 - ((x3 - x6t) - x2))
:((x3t - x4) + x1)
:(x4t + x6ttt + x3tt + x2tt + 2x1tt)
:((10x5 + x8 + 20x2tt + 30x1tt) // 10)
:(2x6 + x7)
:(3x6 + 4x7)
:(x8 - sin(x8))
julia> vars = [:x1, :x2, :x3, :x4, :x6, :x7, :x1t, :x2t, :x3t, :x6t, :x7t, :x6tt, :x7tt];
julia> @time jacobian(eqns, vars) |> Reduce.mat
0.011221 seconds (17.52 k allocations: 1.238 MiB)
8×13 Array{Any,2}:
1 -1 0 0 0 0 0 0 0 0 0 0 0
1 1 -1 0 0 0 0 0 0 1 0 0 0
1 0 0 -1 0 0 0 0 1 0 0 0 0
0 0 0 0 0 0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0 0 0 0 0
0 0 0 0 2 1 0 0 0 0 0 0 0
0 0 0 0 3 4 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0 0 0 0 0As you can see, it's fairly quick to do. I'm not sure how this compares to the performance you were previously getting, my comuter isn't particularly powerful, but you could do about a hundred of these per second on my machine.
If you do the calculation with pre-parsed lists, it is even shorter time
julia> le = Reduce.list(eqns); lv = Reduce.list(vars);
julia> @time mat_jacobian(le, lv)
0.007537 seconds (258 allocations: 24.563 KiB)
[1 -1 0 0 0 0 0 0 0 0 0 0 0]
[1 1 -1 0 0 0 0 0 0 1 0 0 0]
[1 0 0 -1 0 0 0 0 1 0 0 0 0]
[0 0 0 0 0 0 0 0 0 0 0 0 0]
[0 0 0 0 0 0 0 0 0 0 0 0 0]
[0 0 0 0 2 1 0 0 0 0 0 0 0]
[0 0 0 0 3 4 0 0 0 0 0 0 0]
[0 0 0 0 0 0 0 0 0 0 0 0 0]Most of the memory consumption goes into the parsing process, but most of the time consumption is due to the pipe communicating with reduce using the system process. The Reduce software itself does the computation in much less time, but is lagged by the pipe.
And matrix inversion?
That is certainly possible if you are using RExpr objects,
julia> RExpr([:x 1; :y 2])^-1 |> parse |> Reduce.mat
2×2 Array{Any,2}:
:(2 // (2x - y)) :(-1 // (2x - y))
:(-y // (2x - y)) :(x // (2x - y)) However, for this to work natively on Julia matrices, this is part of the same multiple-dispatch optimization that I described above, which is part of changes that I have not commited to github yet, since I need to mak sure that the multiple dispatch is stable, efficient, and doesn't break anything.
Therefore, the built-in inv and ^-1 functions in Julia don't handle this correctly on the current commit of the development branch, but on the next commit this will be included as part of the dispatch.
Perfect, then this is worth a good try.
Are RExpr objects pointers? Are they precompile-safe?
Alright, so I just committed the changes
julia> [:x 1; :y 2]^-1
2×2 Array{Any,2}:
:(2 // (2x - y)) :(-1 // (2x - y))
:(-y // (2x - y)) :(x // (2x - y)) So now the ^-1, ^-2 etc commands work on Symbol and Expr matrices. However, I was not able to figure out yet how to properly overload the inv and \ commands. Tests not added yet.
Are
RExprobjects pointers? Are they precompile-safe?
They are standard Julia objects, they simply contain source code in the REDUCE programming language, the Reduce.jl package uses them throughout to manipulate the commands sent. The parse command converts them into Julia Expr objects. Any of the functions provided by Reduce.jl that can accept Expr and Symbol as input can also take RExpr as input. Sure, they are precompile safe, they merely contain the source code representation of the AST in other language.
They are standard Julia objects, they simply contain source code in the REDUCE programming language
ahh stored as strings, I see. Yup that works.
However, I was not able to figure out yet how to properly overload the inv and \ commands. Tests not added yet.
So you mean, it's just doing the generic route for now but down the line there's a better way to do it strictly in REDUCE?
For inv and \ there is no support currently. I created a prototype function for both, but the multiple dispatch doesn't select them because of how the Types are dispatched for those methods. I couldn't figure out on short notice how that can be resolved, or if it can be resolved at all. The native methods for those use floating point conversions, so you will get an error.
What I'm wondering is whether to use the native generic Julia code or make a single call to Reduce for matrix multiplication and addition. If you have a sparse matrix with mostly numbers and very few Expr objects in it, then it might actually be faster to use the generic algorithm. But if you have a dense matrix with many Expr and Symbol then it would be more efficient to have a single call to Reduce. I'm not sure what the decision should be there, might need some experimentation and depends on what's needed.
A literal ^-1 doesn't lower to inv? Well, it sounds like it works for now and can improve later, that's fine.
Try @edit on a numerical matrix for ^-1 and inv, turns out they are completely different functions.
It just occured to me that the cases for Array{Any,2} should be handled differently from Array{Expr,2} and Array{Symbol,2}. This is because an Any array has the possibility of being sparse in Expr and Symbol objects, while an array of type Expr or Symbol has no chance of being sparse.
This could be used as the criteria for selecting between the generic Julia algorithm with many calls to Reduce or the single call method with one single translation.
Using macros for rcall, I just realized, is much faster than using regular functions.
julia> @time rcall(h)
0.000822 seconds (670 allocations: 34.734 KiB)
:(x ^ 2)
julia> @time @rcall x*log(e^x)
0.000003 seconds (6 allocations: 304 bytes)
:(x ^ 2)It didn't occur to me before that this would be so much faster. REDUCE makes these kind of algebraic calculations in microseconds, so I was wondering why it takes so long for the Pipe to respond. It seems that the communication is more rapid using macros.
julia> h = :(x*log(e^x))
:(x * log(e ^ x))
julia> @eval @time @rcall $h
0.000001 seconds (6 allocations: 304 bytes)
:(x ^ 2)
julia> @time @eval @rcall $h
0.001009 seconds (689 allocations: 36.031 KiB)
:(x ^ 2)Not sure if that's an artifact from these macros or not.
julia> f(h) = @eval @rcall $h
f (generic function with 1 method)
julia> @bench f(h) 1000
0.00037888
julia> @bench rcall(h) 1000
0.000282915This benchmark seems to suggest that rcall is actually faster than f which uses macros.
@time @rcall x*log(e^x) doesn't include the actual compute time because @time gets the evaluated expression. It is equivalent to @time :(x ^ 2).
See the difference below
julia> @macroexpand @time @rcall x*log(e^x)
quote # util.jl, line 235:
local #7#stats = (Base.gc_num)() # util.jl, line 236:
local #9#elapsedtime = (Base.time_ns)() # util.jl, line 237:
local #8#val = $(Expr(:copyast, :($(QuoteNode(:(x ^ 2)))))) # util.jl, line 238:
#9#elapsedtime = (Base.time_ns)() - #9#elapsedtime # util.jl, line 239:
local #10#diff = (Base.GC_Diff)((Base.gc_num)(), #7#stats) # util.jl, line 240:
(Base.time_print)(#9#elapsedtime, #10#diff.allocd, #10#diff.total_time, (Base.gc_alloc_count)(#10#diff)) # util.jl, line 242:
#8#val
end
julia> @macroexpand @time rcall(:(x*log(e^x)))
quote # util.jl, line 235:
local #12#stats = (Base.gc_num)() # util.jl, line 236:
local #14#elapsedtime = (Base.time_ns)() # util.jl, line 237:
local #13#val = rcall($(Expr(:copyast, :($(QuoteNode(:(x * log(e ^ x)))))))) # util.jl, line 238:
#14#elapsedtime = (Base.time_ns)() - #14#elapsedtime # util.jl, line 239:
local #15#diff = (Base.GC_Diff)((Base.gc_num)(), #12#stats) # util.jl, line 240:
(Base.time_print)(#14#elapsedtime, #15#diff.allocd, #15#diff.total_time, (Base.gc_alloc_count)(#15#diff)) # util.jl, line 242:
#13#val
endDue to resolution of issue #8 it is now possible to define inv and \ on symbolic matrices too.