HuggingFace download broken on Julia 1.10 (by ValSplit.jl?)
svilupp opened this issue · comments
J S commented
First of all, thank you for the awesome package!
If I run a snippet from the docs to load bert config from HF Hub (load_config("bert-base-cased")
), I get an error ERROR: MethodError: no method matching iterate(::Nothing)
Possible root cause: I validated that HF Hub, download and parser are working correctly. So I believe ValSplit.jl is the culprit on this line
It works on Julia 1.9 as expected, but not 1.10. It's a shame because transformers loads so much faster on 1.10!
MWE on Julia 1.10
using Transformers
using Transformers.HuggingFace
load_config("bert-base-cased")
# ERROR: MethodError: no method matching iterate(::Nothing)
Full stacktrace
ERROR: MethodError: no method matching iterate(::Nothing)
Closest candidates are:
iterate(::Base.RegexMatchIterator)
@ Base regex.jl:686
iterate(::Base.RegexMatchIterator, ::Any)
@ Base regex.jl:686
iterate(::LLVM.ValueUseSet)
@ LLVM ~/.julia/packages/LLVM/Od0DH/src/core/value.jl:146
...
Stacktrace:
[1] iterate(::Base.Iterators.Filter{Tricks.var"#7#9"{Nothing}, Nothing})
@ Base.Iterators ./iterators.jl:516
[2] iterate(::Base.Generator{Base.Iterators.Filter{Tricks.var"#7#9"{Nothing}, Nothing}, Tricks.var"#6#8"})
@ Base ./generator.jl:44
[3] _collect(::Type{Method}, itr::Base.Generator{Base.Iterators.Filter{Tricks.var"#7#9"{Nothing}, Nothing}, Tricks.var"#6#8"}, isz::Base.SizeUnknown)
@ Base ./array.jl:698
[4] collect(::Type{Method}, itr::Base.Generator{Base.Iterators.Filter{Tricks.var"#7#9"{Nothing}, Nothing}, Tricks.var"#6#8"})
@ Base ./array.jl:692
[5] _methods(f_type::Any, t_type::Any, mod::Nothing)
@ Tricks ~/.julia/packages/Tricks/7oAyo/src/Tricks.jl:146
[6] _methods(f_type::Any, t_type::Any, mod::Nothing)
@ Tricks ~/.julia/packages/Tricks/7oAyo/src/Tricks.jl:143 [inlined]
[7] #s3#3
@ ValSplit ~/.julia/packages/ValSplit/MMCz3/src/ValSplit.jl:42 [inlined]
[8] var"#s3#3"(T::Any, N::Any, P::Any, ::Any, f::Any, types::Any, idx::Any, ptype::Any)
@ ValSplit ./none:0
[9] (::Core.GeneratedFunctionStub)(::UInt64, ::LineNumberNode, ::Any, ::Vararg{Any})
@ Core ./boot.jl:600
[10] valarg_params(f::typeof(config_type), types::Type{Tuple{Val}}, idx::Val{1}, ptype::Type{Symbol})
@ ValSplit ~/.julia/packages/ValSplit/MMCz3/src/ValSplit.jl:60 [inlined]
[11] config_type(model_type::Symbol)
@ Transformers.HuggingFace ~/.julia/packages/ValSplit/MMCz3/src/ValSplit.jl:141
[12] load_config(model_type::Symbol, cfg::JSON3.Object{Vector{UInt8}, Vector{UInt64}})
@ Transformers.HuggingFace ~/.julia/packages/Transformers/qhUJm/src/huggingface/configs/auto.jl:67
[13] _load_config(cfg::JSON3.Object{Vector{UInt8}, Vector{UInt64}})
@ Transformers.HuggingFace ~/.julia/packages/Transformers/qhUJm/src/huggingface/configs/auto.jl:56
[14] load_config(model::String; kw::@Kwargs{})
@ Transformers.HuggingFace ~/.julia/packages/Transformers/qhUJm/src/huggingface/configs/auto.jl:42
[15] top-level scope
@ ~/Documents/Julia-training-range/transformers/first.jl:12
```
Version: Transformers 0.2.7 (latest)
Versioninfo:
> Julia Version 1.10.0-beta1
> Commit 6616549950e (2023-07-25 17:43 UTC)
> Platform Info:
> OS: macOS (arm64-apple-darwin22.4.0)
> CPU: 8 × Apple M1 Pro
> WORD_SIZE: 64
> LIBM: libopenlibm
> LLVM: libLLVM-15.0.7 (ORCJIT, apple-m1)
> Threads: 1 on 6 virtual cores
> Environment:
> JULIA_EDITOR = code
Peter commented
It seems to be caused by ValSplit. We might need to wait until that get fixed.