It seems the nested field of value `nothing` is not supported

[Jun Tian]

MWE:

julia> @option struct OptionA
                         name::Union{Nothing,String}=nothing
                         int::Int = 1
                     end

julia> @option struct OptionB
                         name::Union{Nothing,OptionA}=nothing
                         int::Int = 1
                     end

julia> from_dict(OptionB, Dict{String,Any}("name"=>nothing))
ERROR: option name must be specified using an AbstractDict
Stacktrace:
  [1] error(s::String)
    @ Base ./error.jl:33
  [2] from_kwargs_option_key!(f::Configurations.var"#13#15"{Symbol, Base.Iterators.Pairs{Union{}, Union{}, Tuple{}, NamedTuple{(), Tuple{}}}}, d::Dict{String, Any}, #unused#::Type{OptionA}, name::Symbol, key::Symbol, kw::Base.Iterators.Pairs{Union{}, Union{}, Tuple{}, NamedTuple{(), Tuple{}}})
    @ Configurations ~/.julia/packages/Configurations/CguPm/src/parse.jl:281
  [3] from_kwargs_option_key!(f::Configurations.var"#13#15"{Symbol, Base.Iterators.Pairs{Union{}, Union{}, Tuple{}, NamedTuple{(), Tuple{}}}}, d::Dict{String, Any}, #unused#::Type{Union{Nothing, OptionA}}, name::Symbol, key::Symbol, kw::Base.Iterators.Pairs{Union{}, Union{}, Tuple{}, NamedTuple{(), Tuple{}}})
    @ Configurations ~/.julia/packages/Configurations/CguPm/src/parse.jl:291
  [4] #12
    @ ~/.julia/packages/Configurations/CguPm/src/parse.jl:250 [inlined]
  [5] foreach_keywords!(f::Configurations.var"#12#14"{Base.Iterators.Pairs{Union{}, Union{}, Tuple{}, NamedTuple{(), Tuple{}}}, Dict{String, Any}, Nothing}, list::Dict{String, Any}, #unused#::Type{OptionB})
    @ Configurations ~/.julia/packages/Configurations/CguPm/src/parse.jl:355
  [6] #unsafe_from_underscore_kwargs!#11
    @ ~/.julia/packages/Configurations/CguPm/src/parse.jl:247 [inlined]
  [7] unsafe_from_underscore_kwargs!
    @ ~/.julia/packages/Configurations/CguPm/src/parse.jl:247 [inlined]
  [8] #from_underscore_kwargs!#10
    @ ~/.julia/packages/Configurations/CguPm/src/parse.jl:243 [inlined]
  [9] from_underscore_kwargs! (repeats 2 times)
    @ ~/.julia/packages/Configurations/CguPm/src/parse.jl:242 [inlined]
 [10] from_dict(::Type{OptionB}, d::Dict{String, Any}; kw::Base.Iterators.Pairs{Union{}, Union{}, Tuple{}, NamedTuple{(), Tuple{}}})
    @ Configurations ~/.julia/packages/Configurations/CguPm/src/parse.jl:29
 [11] from_dict(::Type{OptionB}, d::Dict{String, Any})
    @ Configurations ~/.julia/packages/Configurations/CguPm/src/parse.jl:29
 [12] top-level scope
    @ REPL[27]:1

[Xiu-zhe (Roger) Luo]

You are feeding the wrong dict, please verify the input of from_dict with to_dict first.

julia> to_dict(OptionB(name=OptionA(name="aaa")))
OrderedCollections.OrderedDict{String, Any} with 1 entry:
  "name" => OrderedCollections.OrderedDict{String, Any}("name"=>"aaa")

julia> d = to_dict(OptionB(name=OptionA(name="aaa")))
OrderedCollections.OrderedDict{String, Any} with 1 entry:
  "name" => OrderedCollections.OrderedDict{String, Any}("name"=>"aaa")

julia> from_dict(OptionB, d)
OptionB(OptionA("aaa", 1), 1)

The error msg should be improved more on this tho

[Jun Tian]

Hmm, your example works. But the dict passing into the from_dict is extracted from a .yaml file by following https://configurations.rogerluo.dev/stable/quick-start/#Read-from-YAML-files and it is just a plain Dict instead of an OrderedDict, which results to the exception I posted above.

---

I was expecting from_dict(OptionB, Dict{String,Any}("name"=>nothing)) == OptionB(name=nothing)

---

Well, the from_toml works perfectly though.

[Xiu-zhe (Roger) Luo]

Ah right I didn't check this problem on YAML let me chat this again later day. It could be some subtle convention problem with YAML since YAML has explicit null but TOML doesn't.

[Xiu-zhe (Roger) Luo]

Yeah, I can confirm this is because of the convention issue, YAML has explicit null type which will be parse to nothing, however in TOML, if a field is null it should not be specified/empty, thus you will see

julia> to_dict(OptionB())
OrderedCollections.OrderedDict{String, Any}()

as the default output. Actually I also find we have a bug that exclude_nothing=false doesn't work here. The main problem is we are using Maybe{T} = Union{Nothing, T} to annotate Optional fields instead of defining
an explicit Optional type. I'm planning to have an update on this package next month since this month is quite busy for me on a quantum experiment collaboration.

For now, a workaround for this is to manually remove nothings in your dict object to follow "TOML" convention, but I think this should be considered as a bug for sure, since YAML/JSON format is quite important for web services etc.

[Jun Tian]

Thanks for looking into it. No hurry ❤️