Giters

JuliaSymbolics / SymbolicUtils.jl

Symbolic expressions, rewriting and simplification

Home Page:https://docs.sciml.ai/SymbolicUtils/stable/

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

Simplification errors breaking CI for SymbolicRegression.jl

MilesCranmer opened this issue · comments

commented

Hey all,

Not sure what this issue is, and was wondering if you could help me out. Some of my CI tests for SymbolicRegression.jl started failing, and looks to be a simplification error occurring during the search.

Here is the error:

End to end test: Error During Test at C:\Users\runneradmin\.julia\packages\SafeTestsets\A83XK\src\SafeTestsets.jl:25
  Got exception outside of a @test
  LoadError: Failed to apply rule ~x + ~β * ~x => (1 + ~β) * ~x on expression x1^2 + cos(1.3764714x4 - 0.039626822)
Here is the full printout: 
End to end test: Error During Test at C:\Users\runneradmin\.julia\packages\SafeTestsets\A83XK\src\SafeTestsets.jl:25
  Got exception outside of a @test
  LoadError: Failed to apply rule ~x + ~β * ~x => (1 + ~β) * ~x on expression x1^2 + cos(1.3764714x4 - 0.039626822)
  Stacktrace:
    [1] (::Metatheory.Rules.DynamicRule)(term::SymbolicUtils.Term{SymbolicUtils.LiteralReal, Nothing})
      @ Metatheory.Rules C:\Users\runneradmin\.julia\packages\Metatheory\wqlT2\src\Rules.jl:234
    [2] (::SymbolicUtils.ACRule{typeof(Combinatorics.permutations), Metatheory.Rules.DynamicRule})(term::SymbolicUtils.Add{SymbolicUtils.LiteralReal, Int64, Dict{Any, Number}, Nothing})
      @ SymbolicUtils C:\Users\runneradmin\.julia\packages\SymbolicUtils\qulQp\src\rule.jl:59
    [3] (::Metatheory.Rewriters.Chain)(x::SymbolicUtils.Add{SymbolicUtils.LiteralReal, Int64, Dict{Any, Number}, Nothing})
      @ Metatheory.Rewriters C:\Users\runneradmin\.julia\packages\Metatheory\wqlT2\src\Rewriters.jl:69
    [4] (::Metatheory.Rewriters.IfElse{TermInterface.var"#3#4"{typeof(+)}, Metatheory.Rewriters.Chain, Metatheory.Rewriters.Empty})(x::SymbolicUtils.Add{SymbolicUtils.LiteralReal, Int64, Dict{Any, Number}, Nothing})
      @ Metatheory.Rewriters C:\Users\runneradmin\.julia\packages\Metatheory\wqlT2\src\Rewriters.jl:58
    [5] (::Metatheory.Rewriters.Chain)(x::SymbolicUtils.Add{SymbolicUtils.LiteralReal, Int64, Dict{Any, Number}, Nothing})
      @ Metatheory.Rewriters C:\Users\runneradmin\.julia\packages\Metatheory\wqlT2\src\Rewriters.jl:69
    [6] (::Metatheory.Rewriters.RestartedChain{Vector{Metatheory.Rewriters.IfElse{F, Metatheory.Rewriters.Chain, Metatheory.Rewriters.Empty} where F}})(x::SymbolicUtils.Add{SymbolicUtils.LiteralReal, Int64, Dict{Any, Number}, Nothing})
      @ Metatheory.Rewriters C:\Users\runneradmin\.julia\packages\Metatheory\wqlT2\src\Rewriters.jl:89
    [7] (::Metatheory.Rewriters.Chain)(x::SymbolicUtils.Add{SymbolicUtils.LiteralReal, Int64, Dict{Any, Number}, Nothing})
      @ Metatheory.Rewriters C:\Users\runneradmin\.julia\packages\Metatheory\wqlT2\src\Rewriters.jl:69
    [8] (::Metatheory.Rewriters.IfElse{SymbolicUtils.var"#207#290", Metatheory.Rewriters.Chain, Metatheory.Rewriters.IfElse{SymbolicUtils.var"#208#291", Metatheory.Rewriters.Chain, Metatheory.Rewriters.Empty}})(x::SymbolicUtils.Add{SymbolicUtils.LiteralReal, Int64, Dict{Any, Number}, Nothing})
      @ Metatheory.Rewriters C:\Users\runneradmin\.julia\packages\Metatheory\wqlT2\src\Rewriters.jl:58
      @ .\array.jl:716 [inlined]
   [45] map(f::Metatheory.Rewriters.PassThrough{Metatheory.Rewriters.Walk{:post, Metatheory.Rewriters.IfElse{SymbolicUtils.var"#207#290", Metatheory.Rewriters.Chain, Metatheory.Rewriters.IfElse{SymbolicUtils.var"#208#291", Metatheory.Rewriters.Chain, Metatheory.Rewriters.Empty}}, typeof(TermInterface.similarterm), false}}, A::Vector{Any})
      @ Base .\abstractarray.jl:2933
   [46] (::Metatheory.Rewriters.Walk{:post, Metatheory.Rewriters.IfElse{SymbolicUtils.var"#207#290", Metatheory.Rewriters.Chain, Metatheory.Rewriters.IfElse{SymbolicUtils.var"#208#291", Metatheory.Rewriters.Chain, Metatheory.Rewriters.Empty}}, typeof(TermInterface.similarterm), false})(x::SymbolicUtils.Add{SymbolicUtils.LiteralReal, Float32, Dict{Any, Number}, Nothing})
      @ Metatheory.Rewriters C:\Users\runneradmin\.julia\packages\Metatheory\wqlT2\src\Rewriters.jl:198
   [47] (::Metatheory.Rewriters.IfElse{typeof(SymbolicUtils.has_trig_exp), Metatheory.Rewriters.Walk{:post, Metatheory.Rewriters.IfElse{SymbolicUtils.var"#207#290", Metatheory.Rewriters.Chain, Metatheory.Rewriters.IfElse{SymbolicUtils.var"#208#291", Metatheory.Rewriters.Chain, Metatheory.Rewriters.Empty}}, typeof(TermInterface.similarterm), false}, Metatheory.Rewriters.Walk{:post, Metatheory.Rewriters.Chain, typeof(TermInterface.similarterm), false}})(x::SymbolicUtils.Add{SymbolicUtils.LiteralReal, Float32, Dict{Any, Number}, Nothing})
      @ Metatheory.Rewriters C:\Users\runneradmin\.julia\packages\Metatheory\wqlT2\src\Rewriters.jl:58
   [48] (::Metatheory.Rewriters.Fixpoint{Metatheory.Rewriters.IfElse{typeof(SymbolicUtils.has_trig_exp), Metatheory.Rewriters.Walk{:post, Metatheory.Rewriters.IfElse{SymbolicUtils.var"#207#290", Metatheory.Rewriters.Chain, Metatheory.Rewriters.IfElse{SymbolicUtils.var"#208#291", Metatheory.Rewriters.Chain, Metatheory.Rewriters.Empty}}, typeof(TermInterface.similarterm), false}, Metatheory.Rewriters.Walk{:post, Metatheory.Rewriters.Chain, typeof(TermInterface.similarterm), false}}})(x::SymbolicUtils.Term{SymbolicUtils.LiteralReal, Nothing})
      @ Metatheory.Rewriters C:\Users\runneradmin\.julia\packages\Metatheory\wqlT2\src\Rewriters.jl:122
   [49] IfElse
      @ C:\Users\runneradmin\.julia\packages\Metatheory\wqlT2\src\Rewriters.jl:58 [inlined]
   [50] (::Metatheory.Rewriters.PassThrough{Metatheory.Rewriters.IfElse{typeof(TermInterface.istree), Metatheory.Rewriters.Fixpoint{Metatheory.Rewriters.IfElse{typeof(SymbolicUtils.has_trig_exp), Metatheory.Rewriters.Walk{:post, Metatheory.Rewriters.IfElse{SymbolicUtils.var"#207#290", Metatheory.Rewriters.Chain, Metatheory.Rewriters.IfElse{SymbolicUtils.var"#208#291", Metatheory.Rewriters.Chain, Metatheory.Rewriters.Empty}}, typeof(TermInterface.similarterm), false}, Metatheory.Rewriters.Walk{:post, Metatheory.Rewriters.Chain, typeof(TermInterface.similarterm), false}}}, Metatheory.Rewriters.Empty}})(x::SymbolicUtils.Term{SymbolicUtils.LiteralReal, Nothing})
      @ Metatheory.Rewriters C:\Users\runneradmin\.julia\packages\Metatheory\wqlT2\src\Rewriters.jl:188
   [51] simplify(x::SymbolicUtils.Term{SymbolicUtils.LiteralReal, Nothing}; expand::Bool, polynorm::Nothing, threaded::Bool, simplify_fractions::Bool, thread_subtree_cutoff::Int64, rewriter::Nothing)
      @ SymbolicUtils C:\Users\runneradmin\.julia\packages\SymbolicUtils\qulQp\src\simplify.jl:42
   [52] simplify(x::SymbolicUtils.Term{SymbolicUtils.LiteralReal, Nothing})
      @ SymbolicUtils C:\Users\runneradmin\.julia\packages\SymbolicUtils\qulQp\src\simplify.jl:17
   [53] top-level scope
      @ D:\a\SymbolicRegression.jl\SymbolicRegression.jl\example.jl:15
   [54] include(mod::Module, _path::String)
      @ Base .\Base.jl:419
   [55] include(x::String)
      @ Main.var"##383" C:\Users\runneradmin\.julia\packages\SafeTestsets\A83XK\src\SafeTestsets.jl:23
   [56] top-level scope
      @ D:\a\SymbolicRegression.jl\SymbolicRegression.jl\test\full.jl:207
   [57] include(mod::Module, _path::String)
      @ Base .\Base.jl:419
   [58] include(x::String)
      @ Main.var"##383" C:\Users\runneradmin\.julia\packages\SafeTestsets\A83XK\src\SafeTestsets.jl:23
   [59] macro expansion
      @ D:\a\SymbolicRegression.jl\SymbolicRegression.jl\test\runtests.jl:7 [inlined]
   [60] macro expansion
      @ C:\hostedtoolcache\windows\julia\1.8.0\x64\share\julia\stdlib\v1.8\Test\src\Test.jl:1357 [inlined]
   [61] top-level scope
      @ D:\a\SymbolicRegression.jl\SymbolicRegression.jl\test\runtests.jl:7
   [62] eval(m::Module, e::Any)
      @ Core .\boot.jl:368
   [63] top-level scope
      @ C:\Users\runneradmin\.julia\packages\SafeTestsets\A83XK\src\SafeTestsets.jl:23
   [64] include(fname::String)
      @ Base.MainInclude .\client.jl:476
   [65] top-level scope
      @ none:6
   [66] eval
      @ .\boot.jl:368 [inlined]
   [67] exec_options(opts::Base.JLOptions)
      @ Base .\client.jl:276

I unfortunately can't make a MWE at this time, since it seems like

@syms x1 x4
simplify(x1^2 + cos(1.3764714x4 - 0.039626822))

performs as expected. Is there a way you could check that single rule applied to this expression?

This is the job that is failing: https://github.com/MilesCranmer/SymbolicRegression.jl/actions/runs/3192815499/jobs/5210714953. The equation it finds is random each time, so only sometimes it will land this particular equation, hence the other jobs succeeding.

This is another failing example, although it doesn't display exactly why it failed (I presume a similar error?) 
LoadError: LoadError: InterruptException:
  Stacktrace:
    [1] Array
      @ .\boot.jl:448 [inlined]
    [2] vcat(rs::UnitRange{Int64})
      @ Base .\range.jl:1033
    [3] collect
      @ .\range.jl:1043 [inlined]
    [4] iterate
      @ ~\.julia\packages\Combinatorics\Udg6X\src\permutations.jl:43 [inlined]
    [5] (::SymbolicUtils.ACRule{typeof(Combinatorics.permutations), Metatheory.Rules.DynamicRule})(term::SymbolicUtils.Add{SymbolicUtils.LiteralReal, Int64, Dict{Any, Number}, Nothing})
      @ SymbolicUtils ~\.julia\packages\SymbolicUtils\qulQp\src\rule.jl:58
    [6] (::Metatheory.Rewriters.Chain)(x::SymbolicUtils.Add{SymbolicUtils.LiteralReal, Int64, Dict{Any, Number}, Nothing}) (repeats 2 times)
      @ Metatheory.Rewriters ~\.julia\packages\Metatheory\wqlT2\src\Rewriters.jl:69
      @ Main.##326 ~\.julia\packages\SafeTestsets\A83XK\src\SafeTestsets.jl:23
   [40] top-level scope
      @ D:\a\SymbolicRegression.jl\SymbolicRegression.jl\test\full.jl:207
   [41] include(mod::Module, _path::String)
      @ Base .\Base.jl:386
   [42] include(x::String)
      @ Main.##326 ~\.julia\packages\SafeTestsets\A83XK\src\SafeTestsets.jl:23
   [43] macro expansion
      @ D:\a\SymbolicRegression.jl\SymbolicRegression.jl\test\runtests.jl:7 [inlined]
   [44] macro expansion
      @ C:\buildbot\worker\package_win64\build\usr\share\julia\stdlib\v1.6\Test\src\Test.jl:[1151](https://github.com/MilesCranmer/SymbolicRegression.jl/actions/runs/3192688952/jobs/5210447354#step:6:1152) [inlined]
   [45] top-level scope
      @ D:\a\SymbolicRegression.jl\SymbolicRegression.jl\test\runtests.jl:7
   [46] eval(m::Module, e::Any)
      @ Core .\boot.jl:360
   [47] top-level scope
      @ ~\.julia\packages\SafeTestsets\A83XK\src\SafeTestsets.jl:23
   [48] include(fname::String)
      @ Base.MainInclude .\client.jl:444
   [49] top-level scope
      @ none:6
   [50] eval
      @ .\boot.jl:360 [inlined]
   [51] exec_options(opts::Base.JLOptions)
      @ Base .\client.jl:261

(job here: https://github.com/MilesCranmer/SymbolicRegression.jl/actions/runs/3192688952/jobs/5210447354#step:6:1085)

commented

@shashi @YingboMa any help appreciated 🙂

commented

I think something is sending an Interrupt signal...

  caused by: InterruptException:

When a rule fails there are 2 stacktraces ... the second one shows why it failed, the first one just that it did...

commented

Ah, I know why. The job died at exactly 1 h, which is the timeout of my GitHub jobs. So it must have been stuck in a long-running simplification loop.

Is there a way I can specify the maximum number of simplification steps?

commented 
Some rule is reordering the arguments:

```julia
julia> arguments(a)
2-element Vector{Any}:
 -5.8x2
 1.2x2 + cos(3.2x1)

julia> arguments(b)
2-element Vector{Any}:
 1.2x2
 cos(3.2x1) - 5.8x2

incidentally you cannot see it in the printing because we move the negatives to the right for nicer printing:

julia> a
1.2x2 + cos(3.2x1) - 5.8x2

julia> b
1.2x2 + cos(3.2x1) - 5.8x2

So the fixpoint is going into an infinite loop. I was hoping it would be one term with 3 arguments because there is a rule for that.

commented

Seems like it is trying but similarterm is being weird.

julia> SymbolicUtils.similarterm(a, +, [1.2x2, -5.8x2, cos(3.2x1)]) |> arguments
2-element Vector{Any}:
 1.2x2 - 5.8x2
 cos(3.2x1)