MethodError: no method matching length(...)

[Jan Weidner]

I get the following error with both trace and gradtape:

using Ghost
using Yota
using NNlib
f(x,w) = NNlib.conv(x, w)
w = randn(Float32, 1,3,1)
x = randn(Float32, 2,3,1)
@assert f(x,w) isa AbstractArray # ensure input array shapes are compatible

Yota.gradtape(f,x,w) # fails
Ghost.trace(f, x, w) # fails

ERROR: LoadError: MethodError: no method matching length(::NNlib.var"#expand_size#2"{Int64})
Closest candidates are:
  length(::Union{Base.KeySet, Base.ValueIterator}) at abstractdict.jl:58
  length(::Union{DataStructures.OrderedRobinDict, DataStructures.RobinDict}) at /home/jan/.julia/
packages/DataStructures/ixwFs/src/ordered_robin_dict.jl:86
  length(::Union{DataStructures.SortedDict, DataStructures.SortedMultiDict, DataStructures.Sorted
Set}) at /home/jan/.julia/packages/DataStructures/ixwFs/src/container_loops.jl:322
  ...

[Andrei Zhabinski]

I think it's actually the same as #11, e.g. the following gives the wrong output on Ghost@v0.1.0 (borrowed from expand_size):

julia> trace(p -> tuple(p...), (1,))
fn = Core._apply_iterate
((1,), Tape{Dict{Any, Any}}
  inp %1::var"#638#639"
  inp %2::Tuple{Int64}
  # Core._apply_iterate() is missing here
)

The fix for #11 is already merged, but on the main branch there's another (artificial) change that prevents me from checking it. I'll need a bit more time to adjust all the changes.

---

As I mentioned in another issue, you may want to treat functions from NNlib as primitives themselves. The easiest way to do it is to extend the list of primitives e.g. like this:

import Ghost: trace, module_functions, BASE_PRIMITIVE_FUNCTIONS

trace(f, x, w; primitives=vcat(BASE_PRIMITIVE_FUNCTIONS, module_functions(NNlib)))

which gives (tested from the main branch):

(Float32[-1.4576147; 4.51511], Tape{Dict{Any, Any}}
  inp %1::typeof(f)
  inp %2::Array{Float32, 3}
  inp %3::Array{Float32, 3}
  const %4 = conv::typeof(conv)
  %5 = %4(%2, %3)::Array{Float32, 3}
)

[Jan Weidner]

As I mentioned in another issue, you may want to treat functions from NNlib as primitives themselves. The easiest way to do it is to extend the list of primitives e.g. like this:

Yes, I did extend the primitives that's where #11 came from. However, I tried to do it manually, not as elegant as your proposal, thanks a lot! In parallel, I also try what happens when I use the default primitives, that's where this issue came from.
Thanks a lot for working on this!

[Andrei Zhabinski]

#14 fixes the artificial limitation I talked about. However, tracing NNlib.conv() still fails, now with segmentation fault somewhere deep in the Julia compilation/type inference loop in code generated by IRTools (most likely, when expanding the @threads macro in NNlib.conv_im2col!). Eventually, the Compiler Plugins project should make it possible to trace such code as well, but at the moment it's far beyond the scope of Ghost.

Just to close the loop, extending the list of primitives with more high-level functions like NNlib.conv() is still the valid and recommended approach.