MethodError: no method matching iterate(::Core.CodeInfo)
pat-alt opened this issue · comments
I'm trying to use Zygote on Julia v1.6.3, because unfortunately that's all that's currently available on the HPC I'm working on. It fails to precompile throwing the following error:
julia> using Zygote
@Info (Base): Precompiling Zygote [e88e6eb3-aa80-5325-afca-941959d7151f]
│
╰────────────────────────────────────────────────
Fri, 18 Aug 2023 08:51:10
ERROR: LoadError: LoadError: MethodError: no method matching iterate(::Core.CodeInfo)
Closest candidates are:
iterate(::Union{LinRange, StepRangeLen}) at range.jl:664
iterate(::Union{LinRange, StepRangeLen}, ::Int64) at range.jl:664
iterate(::T) where T<:Union{Base.KeySet{var"#s77", var"#s76"} where {var"#s77", var"#s76"<:Dict}, Base.ValueIterator{var"#s75"} where var"#s75"<:Dict} at dict.jl:693
...
Stacktrace:
[1] indexed_iterate(I::Core.CodeInfo, i::Int64)
@ Base ./tuple.jl:89
[2] pullback(::Function, ::Zygote.Context{false}, ::Int64, ::Vararg{Int64, N} where N)
@ Zygote ~/.julia/packages/Zygote/4rucm/src/compiler/interface.jl:44
[3] pullback(::Function, ::Int64, ::Int64)
@ Zygote ~/.julia/packages/Zygote/4rucm/src/compiler/interface.jl:42
[4] gradient(::Function, ::Int64, ::Vararg{Int64, N} where N)
@ Zygote ~/.julia/packages/Zygote/4rucm/src/compiler/interface.jl:96
[5] top-level scope
@ ~/.julia/packages/Zygote/4rucm/src/precompile.jl:17
[6] include(mod::Module, _path::String)
@ Base ./Base.jl:386
[7] include
@ ~/.julia/packages/Zygote/4rucm/src/Zygote.jl:1 [inlined]
[8] macro expansion
@ ~/.julia/packages/Requires/Z8rfN/src/Requires.jl:40 [inlined]
[9] precompile()
@ Zygote ~/.julia/packages/Zygote/4rucm/src/Zygote.jl:63
[10] macro expansion
@ ~/.julia/packages/PrecompileTools/0yi7r/src/workloads.jl:81 [inlined]
[11] top-level scope
@ ~/.julia/packages/Zygote/4rucm/src/Zygote.jl:84
[12] include
@ ./Base.jl:386 [inlined]
[13] include_package_for_output(pkg::Base.PkgId, input::String, depot_path::Vector{String}, dl_load_path::Vector{String}, load_path::Vector{String}, concrete_deps::Vector{Pair{Base.PkgId, UInt64}}, source::Nothing)
@ Base ./loading.jl:1235
[14] top-level scope
@ none:1
[15] eval
@ ./boot.jl:360 [inlined]
[16] eval(x::Expr)
@ Base.MainInclude ./client.jl:446
[17] top-level scope
@ none:1
in expression starting at /Users/FA31DU/.julia/packages/Zygote/4rucm/src/precompile.jl:17
in expression starting at /Users/FA31DU/.julia/packages/Zygote/4rucm/src/Zygote.jl:1
ERROR:
──────────────────────────────────────── ErrorException ────────────────────────────────────────
╭──── Error Stack ─────────────────────────────────────────────────────────────────────────────╮
│ │
│ ──────────────────────────────────── In module Base ──────────────────────────────────── │
│ │
│ ╭───────────────────────────────────────────────────────────────────────────╮ │
│ (1) │ require(into::Module, mod::Symbol) │ │
│ │ ./loading.jl:923 │ │
│ ╰───────────────────────────────────────────────────────────── TOP LEVEL ───╯ │
│ │
│ ──────────────────────────────────────────────────────────────────────────────────────── │
│ Skipped 4 frames in Base │
│ ──────────────────────────────────────────────────────────────────────────────────────── │
│ │
│ ╭───────────────────────────────────────────────────────────────────────────╮ │
│ (6) │ error(s::String) │ │
│ │ ./error.jl:33 │ │
│ ╰──────────────────────────────────────────────────────────── ERROR LINE ───╯ │
│ │
╰──── Error Stack ─────────────────────────────────────────────────────────────────────────────╯
╭─────────────────────────────────────── ErrorException ───────────────────────────────────────╮
│ │
│ Failed to precompile Zygote [e88e6eb3-aa80-5325-afca-941959d7151f] to /Users/FA31DU/.jul │
│ ia/compiled/v1.6/Zygote/jl_chkkNu. │
│ │
╰──────────────────────────────────────────────────────────────────────────────────────────────╯
Is there anything obvious going on here? Is this salvageable or should I just wait for a newer Julia version to be operational on the HPC?
Thanks!
Nothing obvious, I can't think if any scenario where we'd get back a chunk of IR instead of an actual function here. Can you paste the output of Zygote._pullback(Zygote.Context(), pow, 2, 3), where pow can be copied from https://github.com/FluxML/Zygote.jl/blob/master/src/precompile.jl?
I don't think I can properly test this when Zygote fails to precompile? Sorry if I'm missing something
julia> function pow(x, n)
r = 1
while n > 0
n -= 1
r *= x
end
return r
end
pow (generic function with 1 method)
julia> Zygote._pullback(Zygote.Context(), pow, 2, 3)
ERROR:
─────────────────────────────────────────────────────────────── UndefVarError ──────────────────────────────────────────────────────────────
╭──── Error Stack ─────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────╮
│ │
│ ╭───────────────────────────────────────────────────────────────────────────────────────────────────────────────────────╮ │
│ (1) │ top-level scope │ │
│ │ REPL[6]:1 │ │
│ ╰───────────────────────────────────────────────────────────────────────────────────────────────────────── TOP LEVEL ───╯ │
│ │
╰──── Error Stack ─────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────╯
╭────────────────────────────────────────────────────────────── UndefVarError ─────────────────────────────────────────────────────────────╮
│ │
│ UndefVarError: Zygote not defined │
│ │
╰──────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────╯
julia> I've just moved that into the precompile.jl. Here's the output (where the stack trace is the same as in the OP):
julia> using Zygote
@Info (Base): Precompiling Zygote [e88e6eb3-aa80-5325-afca-941959d7151f]
│
╰────────────────────────────────────────────────
Mon, 21 Aug 2023 14:33:39
[ Info: Testing pullback method:
CodeInfo(
1 ─ $(Expr(:meta, :inline))
│ %2 = Base.getfield(args, 1)
│ %3 = Base.getfield(args, 2)
│ %4 = (Vector{Any})()
│ %5 = (Vector{Any})()
│ %6 = (Vector{Any})()
│ %7 = (Vector{Any})()
│ phi_2_1 = %3
│ phi_2_2 = 1
│ phi_2_3 = 0x01
└── goto #2
@ /Users/patrickaltmeyer/.julia/packages/Zygote/4rucm/src/precompile.jl:3 within `pow'
2 ┄ %12 = Zygote._pullback(ctx, Zygote.:>, phi_2_1, 0)
│ %13 = Base.getindex(%12, 1)
│ %14 = Base.getindex(%12, 2)
│ Zygote._push!(%7, phi_2_3)
│ Zygote._push!(%6, %14)
└── goto #5 if not %13
3 ─ goto #4
@ /Users/patrickaltmeyer/.julia/packages/Zygote/4rucm/src/precompile.jl:4 within `pow'
4 ─ %19 = Zygote._pullback(ctx, Zygote.:-, phi_2_1, 1)
│ %20 = Base.getindex(%19, 1)
│ %21 = Base.getindex(%19, 2)
│ @ /Users/patrickaltmeyer/.julia/packages/Zygote/4rucm/src/precompile.jl:5 within `pow'
│ %22 = Zygote._pullback(ctx, Zygote.:*, phi_2_2, %2)
│ %23 = Base.getindex(%22, 1)
│ %24 = Base.getindex(%22, 2)
│ Zygote._push!(%5, %24)
│ Zygote._push!(%4, %21)
│ phi_2_1 = %20
│ phi_2_2 = %23
│ phi_2_3 = 0x02
└── goto #2
5 ─ %31 = Base.tuple(%7, %6, %5, %4)
│ %32 = (Zygote.Pullback{Tuple{typeof(Zygote.pow), Int64, Int64}, Any})(%31)
│ %33 = Base.tuple(phi_2_2, %32)
└── return %33
)
ERROR: LoadError: LoadError: MethodError: no method matching iterate(::Core.CodeInfo)
Closest candidates are:
iterate(::Union{LinRange, StepRangeLen}) at range.jl:664
iterate(::Union{LinRange, StepRangeLen}, ::Int64) at range.jl:664
iterate(::T) where T<:Union{Base.KeySet{var"#s77", var"#s76"} where {var"#s77", var"#s76"<:Dict}, Base.ValueIterator{var"#s75"} where var"#s75"<:Dict} at dict.jl:693
...
Stacktrace:
[...]Do you mind testing on a later 1.6 version? It seems to me that our CI should've caught this if LTS is failing, so perhaps some 1.6 patch release fixed internals so that Zygote works again.
I think the CI runs tests on v1.6.7, which works just fine for me locally:
julia> versioninfo()
Julia Version 1.6.7
Commit 3b76b25b64 (2022-07-19 15:11 UTC)
Platform Info:
OS: macOS (x86_64-apple-darwin21.4.0)
CPU: Intel(R) Core(TM) i9-9880H CPU @ 2.30GHz
WORD_SIZE: 64
LIBM: libopenlibm
LLVM: libLLVM-11.0.1 (ORCJIT, skylake)
Environment:
JULIA_EDITOR = code
julia> using Zygote
julia> That's good. I think it'd probably be easier and more robust to see if you can get a newer patch/minor release of Julia than for us to add a workaround for issues which appear to have been fixed in a patch release. I'll leave this open for a bit though in case anyone does discover the root cause and it happens to be a straightforward fix.
Sounds good, thank you!