Define broadcast for NDArray
vchuravy opened this issue · comments
Right now we are getting a lot of deprecated syntax warnings on v0.6:
WARNING: deprecated syntax "function .+(...)".
Use "function Base.broadcast(::typeof(+), ...)" instead.
In general we should implement broadcast/broadcast! over NDArrays/Symbols exploiting syntactic loop fusion that 0.6 (and v0.5 to some extend) provides.
Custom operators is something I only plan to implement on v0.5 and to run them on the GPU with CUDAnative will require v0.6
From this ref: https://discourse.julialang.org/t/how-to-handle-deprecations-of-new-broadcast-functions/1287 it looks like it can be solved only after corresponding @compat macro is going to be added in Compat.jl, or write custom macro. Are there any other options?
Upd: There as in issue now JuliaLang/Compat.jl#308
We can definitely start thinking how we are going to implement broadcasting in the long run, but you are right this will depend on Compat providing a way of supporting both 0.5 and 0.6
A way forward would be to implement broadcasting over NDArrays by copying the data to the CPU and let Julia perform loop-fusion, later on we can use a similar approach tho GPUArrays.jl to offload kernel functions to the GPU.
More complicated will be supporting SymbolicNodes... ideally we would do something very similar with NativeSymbols, but that is currently not possible and using NativeModules might be to invasive and not good for performance.
Haven't been following Julia broadcast syntax changes yet. It seem to be more complicated than simple syntax sugar that turns a .* b into something like broadcast(*, a, b), as it also deals with more complicated expressions?
But if we could do it that way by properly delegating to the libmxnet broadcasting operator calls (symbolic or ndarray), then I guess it won't be very complicated (such as launching GPU kernels manually).
There is a good overview of what the broadcast syntax changes entail: http://julialang.org/blog/2017/01/moredots
The problem is that X .+ Y .* Z is lowered to broadcast((x,y,z)->x+y*z, X, Y, Z) and works for arbitrary Julia functions.
I chatted with @denizyuret about this and he pointed me to a few resources how other projects solved this.
[1] https://github.com/denizyuret/AutoGrad.jl/blob/master/src/unfuse.jl
[2] https://github.com/MikeInnes/TakingBroadcastSeriously.jl (@MikeInnes)
[3] JuliaLang/julia#22060 (comment) (@ylxdzsw)
