Problem in replicating INVERSE PROBELM tutorial on ODE

Question

Problem in replicating INVERSE PROBELM tutorial on ODE

LorenzoBottaccioli opened this issue 8 months ago · comments

LorenzoBottaccioli commented 8 months ago

Hi I'm trying to replicare the INVERSE PROBLEM for parameter estimation on a simple ODE here the code:

using NeuralPDE, Lux, ModelingToolkit, DataFrames, CSV, DataLoaders, Flux, IntervalSets, Optimization, OptimizationOptimJL

@variables t::Real
@syms T_in(t)::Real  # T_in è ora una variabile di stato da predire
@syms T_ext(t)::Real
@syms Q_heating(t)::Real
@syms Q_cooling(t)::Real
@syms Q_sun(t)::Real
@syms Q_lights(t)::Real
@syms Q_equipment(t)::Real
@parameters R::Real C::Real Cs::Real

Dt = Differential(t)

eqs = [Dt(T_in(t)) ~ (T_ext(t) - T_in(t)) / (R * C) + Q_heating(t) / C - Q_cooling(t) / C + Q_sun(t) / Cs + (Q_lights(t) + Q_equipment(t)) / C]

tspan = (0.0, 365.0 * 24.0 * 60.0)  # Dati per un anno

data = CSV.File("shoebox_free.csv") |> DataFrame

T_in_data = data."OSGB1000005735772_FLOOR_1:Zone Mean Air Temperature [C](TimeStep)"
T_ext_data = data."Environment:Site Outdoor Air Drybulb Temperature [C](TimeStep)"
Q_heating_data = data."OSGB1000005735772_FLOOR_1_HVAC:Zone Ideal Loads Zone Total Heating Rate [W](TimeStep)"
Q_cooling_data = data."OSGB1000005735772_FLOOR_1_HVAC:Zone Ideal Loads Zone Total Cooling Rate [W](TimeStep)"
Q_sun_data = data."Environment:Site Direct Solar Radiation Rate per Area [W/m2](TimeStep)"
Q_lights_data = data."OSGB1000005735772_FLOOR_1:Zone Lights Total Heating Rate [W](TimeStep)"
Q_equipment_data = data."OSGB1000005735772_FLOOR_1:Zone Electric Equipment Total Heating Rate [W](TimeStep)"

t_data = 1:size(data, 1)

dataloader = DataLoader([vcat(T_ext_data, Q_heating_data, Q_cooling_data, Q_sun_data, Q_lights_data, Q_equipment_data, T_in_data)])

input_dim = length(tspan)
hidden_units = 8
chain1 = Lux.Chain(Lux.Dense(input_dim, hidden_units, Lux.σ),
                   Lux.Dense(hidden_units, hidden_units, Lux.σ),
                   Lux.Dense(hidden_units, hidden_units, Lux.σ),
                   Lux.Dense(hidden_units, 1))
dt = 600.0  # 600 secondi (10 minuti)

discretization = NeuralPDE.PhysicsInformedNN([chain1],
                                             NeuralPDE.GridTraining(dt),
                                             param_estim = true)

@named ode_system = NeuralPDE.ODESystem(eqs, [t], [T_in(t), T_ext(t), Q_heating(t), Q_cooling(t), Q_sun(t), Q_lights(t), Q_equipment(t)], [R, C, Cs])

prob = NeuralPDE.discretize(ode_system, discretization)

function additional_loss(phi, θ, p)
    T_in_predict = phi[1](t_data, θ[1])[1]
    return sum(abs2(T_in_predict .- T_in_data) / length(data))
end

res = Optimization.solve(prob, BFGS(), maxiters = 5000, callback = (p, l) -> println("Current loss is: $l"))

p_optimized = res.u[end]

# Plot dei risultati
minimizer = res.u.depvar[1]
T_in_predict = minimizer(t_data)

using Plots
plot(t_data, T_in_data, label = "Dati Osservati")
plot!(t_data, T_in_predict, label = "Temperatura Prevista", linestyle = :dash)

Here the error I get when I run :

@named ode_system = NeuralPDE.ODESystem(eqs, [t], [T_in(t), T_ext(t), Q_heating(t), Q_cooling(t), Q_sun(t), Q_lights(t), Q_equipment(t)], [R, C, Cs])

 MethodError: no method matching -(::Vector{Num}, ::Sym{Real, Base.ImmutableDict{DataType, Any}})

Closest candidates are:
  -(::Term{<:LiteralReal}, ::Sym)
   @ SymbolicUtils C:\Users\loren\.julia\packages\SymbolicUtils\qulQp\src\methods.jl:60
  -(::Array, ::SparseArrays.AbstractSparseMatrixCSC)
   @ SparseArrays C:\Users\loren\AppData\Local\Programs\Julia-1.9.2\share\julia\stdlib\v1.9\SparseArrays\src\sparsematrix.jl:2071
  -(::Array, ::FillArrays.AbstractFill)
   @ FillArrays C:\Users\loren\.julia\packages\FillArrays\KpKaA\src\fillalgebra.jl:250
  ...


Stacktrace:
 [1] is_delay_var(iv::Vector{Num}, var::Term{Real, Nothing})
   @ ModelingToolkit C:\Users\loren\.julia\packages\ModelingToolkit\gZ57L\src\utils.jl:149
 [2] check_variables(dvs::Vector{Term{Real, Nothing}}, iv::Vector{Num})
   @ ModelingToolkit C:\Users\loren\.julia\packages\ModelingToolkit\gZ57L\src\utils.jl:159
 [3] ODESystem(tag::UInt64, deqs::Vector{Equation}, iv::Vector{Num}, dvs::Vector{Term{Real, Nothing}}, ps::Vector{Sym{Real, Base.ImmutableDict{DataType, Any}}}, tspan::Nothing, var_to_name::Dict{Any, Any}, ctrls::Vector{Any}, observed::Vector{Equation}, tgrad::Base.RefValue{Vector{Num}}, jac::Base.RefValue{Any}, ctrl_jac::Base.RefValue{Any}, Wfact::Base.RefValue{Matrix{Num}}, Wfact_t::Base.RefValue{Matrix{Num}}, name::Symbol, systems::Vector{ODESystem}, defaults::Dict{Any, Any}, torn_matching::Nothing, connector_type::Nothing, preface::Nothing, cevents::Vector{ModelingToolkit.SymbolicContinuousCallback}, devents::Vector{ModelingToolkit.SymbolicDiscreteCallback}, metadata::Nothing, tearing_state::Nothing, substitutions::Nothing, complete::Bool, discrete_subsystems::Nothing; checks::Bool)
   @ ModelingToolkit C:\Users\loren\.julia\packages\ModelingToolkit\gZ57L\src\systems\diffeqs\odesystem.jl:141
 [4] ODESystem(deqs::Vector{Equation}, iv::Vector{Num}, dvs::Vector{Num}, ps::Vector{Num}; controls::Vector{Num}, observed::Vector{Equation}, systems::Vector{ODESystem}, tspan::Nothing, name::Symbol, default_u0::Dict{Any, Any}, default_p::Dict{Any, Any}, defaults::Dict{Any, Any}, connector_type::Nothing, preface::Nothing, continuous_events::Nothing, discrete_events::Nothing, checks::Bool, metadata::Nothing)
   @ ModelingToolkit C:\Users\loren\.julia\packages\ModelingToolkit\gZ57L\src\systems\diffeqs\odesystem.jl:204
 [5] top-level scope
   @ C:\Users\loren\.julia\packages\ModelingToolkit\gZ57L\src\systems\abstractsystem.jl:887

Sathvik Bhagavan · Answer 1 · Tue Oct 31 2023 14:44:34 GMT+0800 (China Standard Time)

It should be

@named ode_system = NeuralPDE.ODESystem(eqs, t, [T_in(t), T_ext(t), Q_heating(t), Q_cooling(t), Q_sun(t), Q_lights(t), Q_equipment(t)], [R, C, Cs])

[t] -> t

LorenzoBottaccioli · Answer 2 · Tue Oct 31 2023 16:15:46 GMT+0800 (China Standard Time)

@sathvikbhagavan That solved only that line of code, the script still not working.

MethodError: no method matching discretize(::ODESystem, ::PhysicsInformedNN{GridTraining{Float64}, Nothing, Vector{NeuralPDE.Phi{Lux.Chain{NamedTuple{(:layer_1, :layer_2, :layer_3, :layer_4), Tuple{Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(identity), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}}}}, NamedTuple{(:layer_1, :layer_2, :layer_3, :layer_4), NTuple{4, NamedTuple{(), Tuple{}}}}}}, typeof(NeuralPDE.numeric_derivative), Bool, Nothing, Nothing, Nothing, Base.Pairs{Symbol, Union{}, Tuple{}, NamedTuple{(), Tuple{}}}})

Closest candidates are:
  discretize(::PDESystem, ::PhysicsInformedNN)
   @ NeuralPDE C:\Users\loren\.julia\packages\NeuralPDE\F4RlZ\src\discretize.jl:682


Stacktrace:
 [1] top-level scope
   @ In[9]:1

Sathvik Bhagavan · Answer 3 · Tue Oct 31 2023 16:31:41 GMT+0800 (China Standard Time)

Yes, PhysicsInformedNN only works with PDESystem and not ODESystem.

I am not sure what exactly are you trying to solve here. Can you elaborate?

One way might be to use NNODE by converting ODESystem into ODEProblem

LorenzoBottaccioli · Answer 4 · Tue Oct 31 2023 17:00:21 GMT+0800 (China Standard Time)

@sathvikbhagavan I kneed to discover unknown parameter of an ODE, I was thinking to use the param_estim=true of NeuralPDE.PhysicsInformedNN.
The equation is

[Dt(T_in(t)) ~ (T_ext(t) - T_in(t)) / (R * C) + Q_heating(t) / C - Q_cooling(t) / C + Q_sun(t) / Cs + (Q_lights(t) + Q_equipment(t)) / C]

whereR,C, Csare unknown, al the other are time-dependent variables that are measured every 10 minutes.

Astitva Aggarwal · Answer 5 · Wed Nov 01 2023 22:57:35 GMT+0800 (China Standard Time)

@LorenzoBottaccioli you could set the ODE eqs as a PDEsystem(0 instead of an ODESystem() in your code and have an appropriate additional loss function for parameter estimation in this case, refer Docs.

Sathvik Bhagavan · Answer 6 · Thu Nov 02 2023 00:48:00 GMT+0800 (China Standard Time)

@LorenzoBottaccioli, is it necessary for you to use a PINN? Looking at your problem statement, it seems to be a simple optimization problem for finding parameters. Is there a reason you want to train a PINN as well?

LorenzoBottaccioli · Answer 7 · Thu Nov 02 2023 17:25:33 GMT+0800 (China Standard Time)

@sathvikbhagavan I need an NN to make predictions, this is way I wanted to use a PINN. But I don't have the parameters of the ODE so I need to find them and I was trying to do the things together.
@AstitvaAggarwal I have tried to do what you have suggested already but I still get an error, I have tried to do what is written in the docs but it doesn't work. Here is the code:

using NeuralPDE, Lux, ModelingToolkit, DataFrames, CSV, DataLoaders, Flux, IntervalSets, Optimization, OptimizationOptimJL

# Definisci il modello dell'equazione differenziale
@variables t::Real

@syms T_in(t)::Real
@syms T_ext(t)::Real
@syms Q_heating(t)::Real
@syms Q_cooling(t)::Real
@syms Q_sun(t)::Real
@syms Q_lights(t)::Real
@syms Q_equipment(t)::Real
@parameters R::Real C::Real Cs::Real


Dt = Differential(t)

eqs = [Dt(T_in(t)) ~ (T_ext(t) - T_in(t)) / (R * C) + Q_heating(t) / C - Q_cooling(t) / C + Q_sun(t) / Cs + (Q_lights(t) + Q_equipment(t)) / C]



tspan = (0.0, 365.0 * 24.0 * 60.0)  # Dati per un anno

data = CSV.File("shoebox_free.csv") |> DataFrame

T_ext_data = data."Environment:Site Outdoor Air Drybulb Temperature [C](TimeStep)"
Q_heating_data = data."OSGB1000005735772_FLOOR_1_HVAC:Zone Ideal Loads Zone Total Heating Rate [W](TimeStep)"
Q_cooling_data = data."OSGB1000005735772_FLOOR_1_HVAC:Zone Ideal Loads Zone Total Cooling Rate [W](TimeStep)"
Q_sun_data = data."Environment:Site Direct Solar Radiation Rate per Area [W/m2](TimeStep)"
Q_lights_data = data."OSGB1000005735772_FLOOR_1:Zone Lights Total Heating Rate [W](TimeStep)"
Q_equipment_data = data."OSGB1000005735772_FLOOR_1:Zone Electric Equipment Total Heating Rate [W](TimeStep)"

t_data = 1:size(data, 1)

dataloader = DataLoader([vcat(T_ext_data, Q_heating_data, Q_cooling_data, Q_sun_data, Q_lights_data, Q_equipment_data, t_data)])

input_dim = length(tspan)
hidden_units = 8
chain1 = Lux.Chain(Lux.Dense(input_dim, hidden_units, Lux.σ),
                   Lux.Dense(hidden_units, hidden_units, Lux.σ),
                   Lux.Dense(hidden_units, hidden_units, Lux.σ),
                   Lux.Dense(hidden_units, 1))
dt = 600.0  # 600 secondi (10 minuti)
discretization = NeuralPDE.PhysicsInformedNN([chain1],
                                             NeuralPDE.GridTraining(dt),
                                             param_estim = true)

domains = [t ∈ (0.0,365.0 * 24.0 * 60.0)]
bcs=[Dt(T_in(0))~19.3]

@named pde_system=NeuralPDE.PDESystem(eqs,bcs,domains, [t], [T_in(t), T_ext(t), Q_heating(t), Q_cooling(t), Q_sun(t), Q_lights(t), Q_equipment(t)], [R, C, Cs], defaults = Dict([R => 1.0, C => 1.0, Cs => 1.0]))

prob = NeuralPDE.discretize(pde_system, discretization)

function additional_loss(phi, θ, p)
    T_in_predict = phi[1](t_data, θ[1])[1]
    return sum(abs2(T_in_predict .- T_in_data) / length(data))
end

res = Optimization.solve(prob, BFGS(), maxiters = 5000, callback = (p, l) -> println("Current loss is: $l"))

p_optimized = res.u[end]

minimizer = res.u.depvar[1]
T_in_predict = minimizer(t_data)

using Plots
plot(t_data, T_in_data, label = "Dati Osservati")
plot!(t_data, T_in_predict, label = "Temperatura Prevista", linestyle = :dash)

Here the error:

type NamedTuple has no field T_ext

Stacktrace:
  [1] getproperty
    @ .\Base.jl:37 [inlined]
  [2] getindex(#unused#::ComponentArrays.Axis{(T_in = ViewAxis(1:177, Axis(layer_1 = ViewAxis(1:24, Axis(weight = ViewAxis(1:16, ShapedAxis((8, 2), NamedTuple())), bias = ViewAxis(17:24, ShapedAxis((8, 1), NamedTuple())))), layer_2 = ViewAxis(25:96, Axis(weight = ViewAxis(1:64, ShapedAxis((8, 8), NamedTuple())), bias = ViewAxis(65:72, ShapedAxis((8, 1), NamedTuple())))), layer_3 = ViewAxis(97:168, Axis(weight = ViewAxis(1:64, ShapedAxis((8, 8), NamedTuple())), bias = ViewAxis(65:72, ShapedAxis((8, 1), NamedTuple())))), layer_4 = ViewAxis(169:177, Axis(weight = ViewAxis(1:8, ShapedAxis((1, 8), NamedTuple())), bias = ViewAxis(9:9, ShapedAxis((1, 1), NamedTuple())))))),)}, s::Symbol)
    @ ComponentArrays C:\Users\loren\.julia\packages\ComponentArrays\Cy8Rn\src\axis.jl:157
  [3] _broadcast_getindex_evalf
    @ .\broadcast.jl:683 [inlined]
  [4] _broadcast_getindex
    @ .\broadcast.jl:656 [inlined]
  [5] (::Base.Broadcast.var"#31#32"{Base.Broadcast.Broadcasted{Base.Broadcast.Style{Tuple}, Nothing, typeof(getindex), Tuple{Tuple{ComponentArrays.Axis{(T_in = ViewAxis(1:177, Axis(layer_1 = ViewAxis(1:24, Axis(weight = ViewAxis(1:16, ShapedAxis((8, 2), NamedTuple())), bias = ViewAxis(17:24, ShapedAxis((8, 1), NamedTuple())))), layer_2 = ViewAxis(25:96, Axis(weight = ViewAxis(1:64, ShapedAxis((8, 8), NamedTuple())), bias = ViewAxis(65:72, ShapedAxis((8, 1), NamedTuple())))), layer_3 = ViewAxis(97:168, Axis(weight = ViewAxis(1:64, ShapedAxis((8, 8), NamedTuple())), bias = ViewAxis(65:72, ShapedAxis((8, 1), NamedTuple())))), layer_4 = ViewAxis(169:177, Axis(weight = ViewAxis(1:8, ShapedAxis((1, 8), NamedTuple())), bias = ViewAxis(9:9, ShapedAxis((1, 1), NamedTuple())))))),)}}, Base.Broadcast.Broadcasted{Base.Broadcast.Style{Tuple}, Nothing, typeof(ComponentArrays.getval), Tuple{Tuple{DataType}}}}}})(k::Int64)
    @ Base.Broadcast .\broadcast.jl:1088
  [6] ntuple
    @ .\ntuple.jl:48 [inlined]
  [7] copy
    @ .\broadcast.jl:1088 [inlined]
  [8] materialize(bc::Base.Broadcast.Broadcasted{Base.Broadcast.Style{Tuple}, Nothing, typeof(getindex), Tuple{Tuple{ComponentArrays.Axis{(T_in = ViewAxis(1:177, Axis(layer_1 = ViewAxis(1:24, Axis(weight = ViewAxis(1:16, ShapedAxis((8, 2), NamedTuple())), bias = ViewAxis(17:24, ShapedAxis((8, 1), NamedTuple())))), layer_2 = ViewAxis(25:96, Axis(weight = ViewAxis(1:64, ShapedAxis((8, 8), NamedTuple())), bias = ViewAxis(65:72, ShapedAxis((8, 1), NamedTuple())))), layer_3 = ViewAxis(97:168, Axis(weight = ViewAxis(1:64, ShapedAxis((8, 8), NamedTuple())), bias = ViewAxis(65:72, ShapedAxis((8, 1), NamedTuple())))), layer_4 = ViewAxis(169:177, Axis(weight = ViewAxis(1:8, ShapedAxis((1, 8), NamedTuple())), bias = ViewAxis(9:9, ShapedAxis((1, 1), NamedTuple())))))),)}}, Base.Broadcast.Broadcasted{Base.Broadcast.Style{Tuple}, Nothing, typeof(ComponentArrays.getval), Tuple{Tuple{DataType}}}}})
    @ Base.Broadcast .\broadcast.jl:873
  [9] #s20#55
    @ C:\Users\loren\.julia\packages\ComponentArrays\Cy8Rn\src\array_interface.jl:122 [inlined]
 [10] var"#s20#55"(::Any, index_fun::Any, x::Any, idx::Any)
    @ ComponentArrays .\none:0
 [11] (::Core.GeneratedFunctionStub)(::Any, ::Vararg{Any})
    @ Core .\boot.jl:602
 [12] getproperty
    @ C:\Users\loren\.julia\packages\ComponentArrays\Cy8Rn\src\namedtuple_interface.jl:14 [inlined]
 [13] rrule(#unused#::typeof(getproperty), x::ComponentArrays.ComponentVector{Float64, SubArray{Float64, 1, Vector{Float64}, Tuple{UnitRange{Int64}}, true}, Tuple{ComponentArrays.Axis{(T_in = ViewAxis(1:177, Axis(layer_1 = ViewAxis(1:24, Axis(weight = ViewAxis(1:16, ShapedAxis((8, 2), NamedTuple())), bias = ViewAxis(17:24, ShapedAxis((8, 1), NamedTuple())))), layer_2 = ViewAxis(25:96, Axis(weight = ViewAxis(1:64, ShapedAxis((8, 8), NamedTuple())), bias = ViewAxis(65:72, ShapedAxis((8, 1), NamedTuple())))), layer_3 = ViewAxis(97:168, Axis(weight = ViewAxis(1:64, ShapedAxis((8, 8), NamedTuple())), bias = ViewAxis(65:72, ShapedAxis((8, 1), NamedTuple())))), layer_4 = ViewAxis(169:177, Axis(weight = ViewAxis(1:8, ShapedAxis((1, 8), NamedTuple())), bias = ViewAxis(9:9, ShapedAxis((1, 1), NamedTuple())))))),)}}}, s::Symbol)
    @ ComponentArrays C:\Users\loren\.julia\packages\ComponentArrays\Cy8Rn\src\compat\chainrulescore.jl:8
 [14] rrule
    @ C:\Users\loren\.julia\packages\ChainRulesCore\7MWx2\src\rules.jl:134 [inlined]
 [15] chain_rrule
    @ C:\Users\loren\.julia\packages\Zygote\YYT6v\src\compiler\chainrules.jl:223 [inlined]
 [16] macro expansion
    @ C:\Users\loren\.julia\packages\Zygote\YYT6v\src\compiler\interface2.jl:101 [inlined]
 [17] _pullback
    @ C:\Users\loren\.julia\packages\Zygote\YYT6v\src\compiler\interface2.jl:101 [inlined]
 [18] macro expansion
    @ C:\Users\loren\.julia\packages\Zygote\YYT6v\src\lib\literal_getproperty.jl:93 [inlined]
 [19] _pullback(ctx::Zygote.Context{false}, #unused#::typeof(ZygoteRules.literal_getproperty), x::ComponentArrays.ComponentVector{Float64, SubArray{Float64, 1, Vector{Float64}, Tuple{UnitRange{Int64}}, true}, Tuple{ComponentArrays.Axis{(T_in = ViewAxis(1:177, Axis(layer_1 = ViewAxis(1:24, Axis(weight = ViewAxis(1:16, ShapedAxis((8, 2), NamedTuple())), bias = ViewAxis(17:24, ShapedAxis((8, 1), NamedTuple())))), layer_2 = ViewAxis(25:96, Axis(weight = ViewAxis(1:64, ShapedAxis((8, 8), NamedTuple())), bias = ViewAxis(65:72, ShapedAxis((8, 1), NamedTuple())))), layer_3 = ViewAxis(97:168, Axis(weight = ViewAxis(1:64, ShapedAxis((8, 8), NamedTuple())), bias = ViewAxis(65:72, ShapedAxis((8, 1), NamedTuple())))), layer_4 = ViewAxis(169:177, Axis(weight = ViewAxis(1:8, ShapedAxis((1, 8), NamedTuple())), bias = ViewAxis(9:9, ShapedAxis((1, 1), NamedTuple())))))),)}}}, f::Val{:T_ext})
    @ Zygote C:\Users\loren\.julia\packages\Zygote\YYT6v\src\lib\literal_getproperty.jl:117
 [20] macro expansion
    @ C:\Users\loren\.julia\packages\NeuralPDE\F4RlZ\src\discretize.jl:169 [inlined]
 [21] macro expansion
    @ C:\Users\loren\.julia\packages\RuntimeGeneratedFunctions\Yo8zx\src\RuntimeGeneratedFunctions.jl:163 [inlined]
 [22] macro expansion
    @ .\none:0 [inlined]
 [23] _pullback
    @ .\none:0 [inlined]
 [24] _pullback(::Zygote.Context{false}, ::typeof(RuntimeGeneratedFunctions.generated_callfunc), ::RuntimeGeneratedFunctions.RuntimeGeneratedFunction{(:cord, Symbol("##θ#293"), :phi, :derivative, :integral, :u, :p), NeuralPDE.var"#_RGF_ModTag", NeuralPDE.var"#_RGF_ModTag", (0xb5fb80bb, 0x61a62589, 0x2ec55669, 0x07bea97e, 0xb7ae3a52), Expr}, ::Matrix{Float64}, ::ComponentArrays.ComponentVector{Float64, Vector{Float64}, Tuple{ComponentArrays.Axis{(depvar = ViewAxis(1:177, Axis(T_in = ViewAxis(1:177, Axis(layer_1 = ViewAxis(1:24, Axis(weight = ViewAxis(1:16, ShapedAxis((8, 2), NamedTuple())), bias = ViewAxis(17:24, ShapedAxis((8, 1), NamedTuple())))), layer_2 = ViewAxis(25:96, Axis(weight = ViewAxis(1:64, ShapedAxis((8, 8), NamedTuple())), bias = ViewAxis(65:72, ShapedAxis((8, 1), NamedTuple())))), layer_3 = ViewAxis(97:168, Axis(weight = ViewAxis(1:64, ShapedAxis((8, 8), NamedTuple())), bias = ViewAxis(65:72, ShapedAxis((8, 1), NamedTuple())))), layer_4 = ViewAxis(169:177, Axis(weight = ViewAxis(1:8, ShapedAxis((1, 8), NamedTuple())), bias = ViewAxis(9:9, ShapedAxis((1, 1), NamedTuple())))))),)), p = 178:180)}}}, ::Vector{NeuralPDE.Phi{Lux.Chain{NamedTuple{(:layer_1, :layer_2, :layer_3, :layer_4), Tuple{Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(identity), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}}}}, NamedTuple{(:layer_1, :layer_2, :layer_3, :layer_4), NTuple{4, NamedTuple{(), Tuple{}}}}}}, ::typeof(NeuralPDE.numeric_derivative), ::NeuralPDE.var"#269#276"{NeuralPDE.var"#269#270#277"{typeof(NeuralPDE.numeric_derivative)}, Dict{Symbol, Int64}, Dict{Symbol, Int64}, GridTraining{Float64}}, ::NeuralPDE.var"#12#13", ::Vector{Float64})
    @ Zygote C:\Users\loren\.julia\packages\Zygote\YYT6v\src\compiler\interface2.jl:0
 [25] _apply(::Function, ::Vararg{Any})
    @ Core .\boot.jl:838
 [26] adjoint
    @ C:\Users\loren\.julia\packages\Zygote\YYT6v\src\lib\lib.jl:203 [inlined]
 [27] _pullback
    @ C:\Users\loren\.julia\packages\ZygoteRules\4nXuu\src\adjoint.jl:66 [inlined]
 [28] _pullback
    @ C:\Users\loren\.julia\packages\RuntimeGeneratedFunctions\Yo8zx\src\RuntimeGeneratedFunctions.jl:150 [inlined]
 [29] _pullback(::Zygote.Context{false}, ::RuntimeGeneratedFunctions.RuntimeGeneratedFunction{(:cord, Symbol("##θ#293"), :phi, :derivative, :integral, :u, :p), NeuralPDE.var"#_RGF_ModTag", NeuralPDE.var"#_RGF_ModTag", (0xb5fb80bb, 0x61a62589, 0x2ec55669, 0x07bea97e, 0xb7ae3a52), Expr}, ::Matrix{Float64}, ::ComponentArrays.ComponentVector{Float64, Vector{Float64}, Tuple{ComponentArrays.Axis{(depvar = ViewAxis(1:177, Axis(T_in = ViewAxis(1:177, Axis(layer_1 = ViewAxis(1:24, Axis(weight = ViewAxis(1:16, ShapedAxis((8, 2), NamedTuple())), bias = ViewAxis(17:24, ShapedAxis((8, 1), NamedTuple())))), layer_2 = ViewAxis(25:96, Axis(weight = ViewAxis(1:64, ShapedAxis((8, 8), NamedTuple())), bias = ViewAxis(65:72, ShapedAxis((8, 1), NamedTuple())))), layer_3 = ViewAxis(97:168, Axis(weight = ViewAxis(1:64, ShapedAxis((8, 8), NamedTuple())), bias = ViewAxis(65:72, ShapedAxis((8, 1), NamedTuple())))), layer_4 = ViewAxis(169:177, Axis(weight = ViewAxis(1:8, ShapedAxis((1, 8), NamedTuple())), bias = ViewAxis(9:9, ShapedAxis((1, 1), NamedTuple())))))),)), p = 178:180)}}}, ::Vector{NeuralPDE.Phi{Lux.Chain{NamedTuple{(:layer_1, :layer_2, :layer_3, :layer_4), Tuple{Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(identity), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}}}}, NamedTuple{(:layer_1, :layer_2, :layer_3, :layer_4), NTuple{4, NamedTuple{(), Tuple{}}}}}}, ::typeof(NeuralPDE.numeric_derivative), ::NeuralPDE.var"#269#276"{NeuralPDE.var"#269#270#277"{typeof(NeuralPDE.numeric_derivative)}, Dict{Symbol, Int64}, Dict{Symbol, Int64}, GridTraining{Float64}}, ::NeuralPDE.var"#12#13", ::Vector{Float64})
    @ Zygote C:\Users\loren\.julia\packages\Zygote\YYT6v\src\compiler\interface2.jl:0
 [30] _pullback
    @ C:\Users\loren\.julia\packages\NeuralPDE\F4RlZ\src\discretize.jl:192 [inlined]
 [31] _pullback(::Zygote.Context{false}, ::NeuralPDE.var"#209#210"{RuntimeGeneratedFunctions.RuntimeGeneratedFunction{(:cord, Symbol("##θ#293"), :phi, :derivative, :integral, :u, :p), NeuralPDE.var"#_RGF_ModTag", NeuralPDE.var"#_RGF_ModTag", (0xb5fb80bb, 0x61a62589, 0x2ec55669, 0x07bea97e, 0xb7ae3a52), Expr}, NeuralPDE.var"#12#13", NeuralPDE.var"#269#276"{NeuralPDE.var"#269#270#277"{typeof(NeuralPDE.numeric_derivative)}, Dict{Symbol, Int64}, Dict{Symbol, Int64}, GridTraining{Float64}}, typeof(NeuralPDE.numeric_derivative), Vector{NeuralPDE.Phi{Lux.Chain{NamedTuple{(:layer_1, :layer_2, :layer_3, :layer_4), Tuple{Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(identity), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}}}}, NamedTuple{(:layer_1, :layer_2, :layer_3, :layer_4), NTuple{4, NamedTuple{(), Tuple{}}}}}}, Vector{Float64}}, ::Matrix{Float64}, ::ComponentArrays.ComponentVector{Float64, Vector{Float64}, Tuple{ComponentArrays.Axis{(depvar = ViewAxis(1:177, Axis(T_in = ViewAxis(1:177, Axis(layer_1 = ViewAxis(1:24, Axis(weight = ViewAxis(1:16, ShapedAxis((8, 2), NamedTuple())), bias = ViewAxis(17:24, ShapedAxis((8, 1), NamedTuple())))), layer_2 = ViewAxis(25:96, Axis(weight = ViewAxis(1:64, ShapedAxis((8, 8), NamedTuple())), bias = ViewAxis(65:72, ShapedAxis((8, 1), NamedTuple())))), layer_3 = ViewAxis(97:168, Axis(weight = ViewAxis(1:64, ShapedAxis((8, 8), NamedTuple())), bias = ViewAxis(65:72, ShapedAxis((8, 1), NamedTuple())))), layer_4 = ViewAxis(169:177, Axis(weight = ViewAxis(1:8, ShapedAxis((1, 8), NamedTuple())), bias = ViewAxis(9:9, ShapedAxis((1, 1), NamedTuple())))))),)), p = 178:180)}}})
    @ Zygote C:\Users\loren\.julia\packages\Zygote\YYT6v\src\compiler\interface2.jl:0
 [32] _pullback
    @ C:\Users\loren\.julia\packages\NeuralPDE\F4RlZ\src\training_strategies.jl:48 [inlined]
 [33] _pullback(ctx::Zygote.Context{false}, f::NeuralPDE.var"#74#75"{NeuralPDE.var"#209#210"{RuntimeGeneratedFunctions.RuntimeGeneratedFunction{(:cord, Symbol("##θ#293"), :phi, :derivative, :integral, :u, :p), NeuralPDE.var"#_RGF_ModTag", NeuralPDE.var"#_RGF_ModTag", (0xb5fb80bb, 0x61a62589, 0x2ec55669, 0x07bea97e, 0xb7ae3a52), Expr}, NeuralPDE.var"#12#13", NeuralPDE.var"#269#276"{NeuralPDE.var"#269#270#277"{typeof(NeuralPDE.numeric_derivative)}, Dict{Symbol, Int64}, Dict{Symbol, Int64}, GridTraining{Float64}}, typeof(NeuralPDE.numeric_derivative), Vector{NeuralPDE.Phi{Lux.Chain{NamedTuple{(:layer_1, :layer_2, :layer_3, :layer_4), Tuple{Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(identity), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}}}}, NamedTuple{(:layer_1, :layer_2, :layer_3, :layer_4), NTuple{4, NamedTuple{(), Tuple{}}}}}}, Vector{Float64}}, Matrix{Float64}}, args::ComponentArrays.ComponentVector{Float64, Vector{Float64}, Tuple{ComponentArrays.Axis{(depvar = ViewAxis(1:177, Axis(T_in = ViewAxis(1:177, Axis(layer_1 = ViewAxis(1:24, Axis(weight = ViewAxis(1:16, ShapedAxis((8, 2), NamedTuple())), bias = ViewAxis(17:24, ShapedAxis((8, 1), NamedTuple())))), layer_2 = ViewAxis(25:96, Axis(weight = ViewAxis(1:64, ShapedAxis((8, 8), NamedTuple())), bias = ViewAxis(65:72, ShapedAxis((8, 1), NamedTuple())))), layer_3 = ViewAxis(97:168, Axis(weight = ViewAxis(1:64, ShapedAxis((8, 8), NamedTuple())), bias = ViewAxis(65:72, ShapedAxis((8, 1), NamedTuple())))), layer_4 = ViewAxis(169:177, Axis(weight = ViewAxis(1:8, ShapedAxis((1, 8), NamedTuple())), bias = ViewAxis(9:9, ShapedAxis((1, 1), NamedTuple())))))),)), p = 178:180)}}})
    @ Zygote C:\Users\loren\.julia\packages\Zygote\YYT6v\src\compiler\interface2.jl:0
 [34] _pullback
    @ .\none:0 [inlined]
 [35] _pullback(ctx::Zygote.Context{false}, f::NeuralPDE.var"#294#309"{ComponentArrays.ComponentVector{Float64, Vector{Float64}, Tuple{ComponentArrays.Axis{(depvar = ViewAxis(1:177, Axis(T_in = ViewAxis(1:177, Axis(layer_1 = ViewAxis(1:24, Axis(weight = ViewAxis(1:16, ShapedAxis((8, 2), NamedTuple())), bias = ViewAxis(17:24, ShapedAxis((8, 1), NamedTuple())))), layer_2 = ViewAxis(25:96, Axis(weight = ViewAxis(1:64, ShapedAxis((8, 8), NamedTuple())), bias = ViewAxis(65:72, ShapedAxis((8, 1), NamedTuple())))), layer_3 = ViewAxis(97:168, Axis(weight = ViewAxis(1:64, ShapedAxis((8, 8), NamedTuple())), bias = ViewAxis(65:72, ShapedAxis((8, 1), NamedTuple())))), layer_4 = ViewAxis(169:177, Axis(weight = ViewAxis(1:8, ShapedAxis((1, 8), NamedTuple())), bias = ViewAxis(9:9, ShapedAxis((1, 1), NamedTuple())))))),)), p = 178:180)}}}}, args::NeuralPDE.var"#74#75"{NeuralPDE.var"#209#210"{RuntimeGeneratedFunctions.RuntimeGeneratedFunction{(:cord, Symbol("##θ#293"), :phi, :derivative, :integral, :u, :p), NeuralPDE.var"#_RGF_ModTag", NeuralPDE.var"#_RGF_ModTag", (0xb5fb80bb, 0x61a62589, 0x2ec55669, 0x07bea97e, 0xb7ae3a52), Expr}, NeuralPDE.var"#12#13", NeuralPDE.var"#269#276"{NeuralPDE.var"#269#270#277"{typeof(NeuralPDE.numeric_derivative)}, Dict{Symbol, Int64}, Dict{Symbol, Int64}, GridTraining{Float64}}, typeof(NeuralPDE.numeric_derivative), Vector{NeuralPDE.Phi{Lux.Chain{NamedTuple{(:layer_1, :layer_2, :layer_3, :layer_4), Tuple{Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(identity), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}}}}, NamedTuple{(:layer_1, :layer_2, :layer_3, :layer_4), NTuple{4, NamedTuple{(), Tuple{}}}}}}, Vector{Float64}}, Matrix{Float64}})
    @ Zygote C:\Users\loren\.julia\packages\Zygote\YYT6v\src\compiler\interface2.jl:0
 [36] #661
    @ C:\Users\loren\.julia\packages\Zygote\YYT6v\src\lib\array.jl:183 [inlined]
 [37] iterate
    @ .\generator.jl:47 [inlined]
 [38] _collect(c::Vector{NeuralPDE.var"#74#75"{NeuralPDE.var"#209#210"{RuntimeGeneratedFunctions.RuntimeGeneratedFunction{(:cord, Symbol("##θ#293"), :phi, :derivative, :integral, :u, :p), NeuralPDE.var"#_RGF_ModTag", NeuralPDE.var"#_RGF_ModTag", (0xb5fb80bb, 0x61a62589, 0x2ec55669, 0x07bea97e, 0xb7ae3a52), Expr}, NeuralPDE.var"#12#13", NeuralPDE.var"#269#276"{NeuralPDE.var"#269#270#277"{typeof(NeuralPDE.numeric_derivative)}, Dict{Symbol, Int64}, Dict{Symbol, Int64}, GridTraining{Float64}}, typeof(NeuralPDE.numeric_derivative), Vector{NeuralPDE.Phi{Lux.Chain{NamedTuple{(:layer_1, :layer_2, :layer_3, :layer_4), Tuple{Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(identity), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}}}}, NamedTuple{(:layer_1, :layer_2, :layer_3, :layer_4), NTuple{4, NamedTuple{(), Tuple{}}}}}}, Vector{Float64}}, Matrix{Float64}}}, itr::Base.Generator{Vector{NeuralPDE.var"#74#75"{NeuralPDE.var"#209#210"{RuntimeGeneratedFunctions.RuntimeGeneratedFunction{(:cord, Symbol("##θ#293"), :phi, :derivative, :integral, :u, :p), NeuralPDE.var"#_RGF_ModTag", NeuralPDE.var"#_RGF_ModTag", (0xb5fb80bb, 0x61a62589, 0x2ec55669, 0x07bea97e, 0xb7ae3a52), Expr}, NeuralPDE.var"#12#13", NeuralPDE.var"#269#276"{NeuralPDE.var"#269#270#277"{typeof(NeuralPDE.numeric_derivative)}, Dict{Symbol, Int64}, Dict{Symbol, Int64}, GridTraining{Float64}}, typeof(NeuralPDE.numeric_derivative), Vector{NeuralPDE.Phi{Lux.Chain{NamedTuple{(:layer_1, :layer_2, :layer_3, :layer_4), Tuple{Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(identity), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}}}}, NamedTuple{(:layer_1, :layer_2, :layer_3, :layer_4), NTuple{4, NamedTuple{(), Tuple{}}}}}}, Vector{Float64}}, Matrix{Float64}}}, Zygote.var"#661#665"{Zygote.Context{false}, NeuralPDE.var"#294#309"{ComponentArrays.ComponentVector{Float64, Vector{Float64}, Tuple{ComponentArrays.Axis{(depvar = ViewAxis(1:177, Axis(T_in = ViewAxis(1:177, Axis(layer_1 = ViewAxis(1:24, Axis(weight = ViewAxis(1:16, ShapedAxis((8, 2), NamedTuple())), bias = ViewAxis(17:24, ShapedAxis((8, 1), NamedTuple())))), layer_2 = ViewAxis(25:96, Axis(weight = ViewAxis(1:64, ShapedAxis((8, 8), NamedTuple())), bias = ViewAxis(65:72, ShapedAxis((8, 1), NamedTuple())))), layer_3 = ViewAxis(97:168, Axis(weight = ViewAxis(1:64, ShapedAxis((8, 8), NamedTuple())), bias = ViewAxis(65:72, ShapedAxis((8, 1), NamedTuple())))), layer_4 = ViewAxis(169:177, Axis(weight = ViewAxis(1:8, ShapedAxis((1, 8), NamedTuple())), bias = ViewAxis(9:9, ShapedAxis((1, 1), NamedTuple())))))),)), p = 178:180)}}}}}}, #unused#::Base.EltypeUnknown, isz::Base.HasShape{1})
    @ Base .\array.jl:802
 [39] collect_similar
    @ .\array.jl:711 [inlined]
 [40] map
    @ .\abstractarray.jl:3261 [inlined]
 [41] ∇map(cx::Zygote.Context{false}, f::NeuralPDE.var"#294#309"{ComponentArrays.ComponentVector{Float64, Vector{Float64}, Tuple{ComponentArrays.Axis{(depvar = ViewAxis(1:177, Axis(T_in = ViewAxis(1:177, Axis(layer_1 = ViewAxis(1:24, Axis(weight = ViewAxis(1:16, ShapedAxis((8, 2), NamedTuple())), bias = ViewAxis(17:24, ShapedAxis((8, 1), NamedTuple())))), layer_2 = ViewAxis(25:96, Axis(weight = ViewAxis(1:64, ShapedAxis((8, 8), NamedTuple())), bias = ViewAxis(65:72, ShapedAxis((8, 1), NamedTuple())))), layer_3 = ViewAxis(97:168, Axis(weight = ViewAxis(1:64, ShapedAxis((8, 8), NamedTuple())), bias = ViewAxis(65:72, ShapedAxis((8, 1), NamedTuple())))), layer_4 = ViewAxis(169:177, Axis(weight = ViewAxis(1:8, ShapedAxis((1, 8), NamedTuple())), bias = ViewAxis(9:9, ShapedAxis((1, 1), NamedTuple())))))),)), p = 178:180)}}}}, args::Vector{NeuralPDE.var"#74#75"{NeuralPDE.var"#209#210"{RuntimeGeneratedFunctions.RuntimeGeneratedFunction{(:cord, Symbol("##θ#293"), :phi, :derivative, :integral, :u, :p), NeuralPDE.var"#_RGF_ModTag", NeuralPDE.var"#_RGF_ModTag", (0xb5fb80bb, 0x61a62589, 0x2ec55669, 0x07bea97e, 0xb7ae3a52), Expr}, NeuralPDE.var"#12#13", NeuralPDE.var"#269#276"{NeuralPDE.var"#269#270#277"{typeof(NeuralPDE.numeric_derivative)}, Dict{Symbol, Int64}, Dict{Symbol, Int64}, GridTraining{Float64}}, typeof(NeuralPDE.numeric_derivative), Vector{NeuralPDE.Phi{Lux.Chain{NamedTuple{(:layer_1, :layer_2, :layer_3, :layer_4), Tuple{Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(identity), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}}}}, NamedTuple{(:layer_1, :layer_2, :layer_3, :layer_4), NTuple{4, NamedTuple{(), Tuple{}}}}}}, Vector{Float64}}, Matrix{Float64}}})
    @ Zygote C:\Users\loren\.julia\packages\Zygote\YYT6v\src\lib\array.jl:183
 [42] _pullback
    @ C:\Users\loren\.julia\packages\Zygote\YYT6v\src\lib\array.jl:226 [inlined]
 [43] _pullback
    @ C:\Users\loren\.julia\packages\NeuralPDE\F4RlZ\src\discretize.jl:592 [inlined]
 [44] _pullback(::Zygote.Context{false}, ::NeuralPDE.var"#full_loss_function#308"{NeuralPDE.var"#null_nonadaptive_loss#115", Vector{NeuralPDE.var"#74#75"{NeuralPDE.var"#209#210"{RuntimeGeneratedFunctions.RuntimeGeneratedFunction{(:cord, Symbol("##θ#293"), :phi, :derivative, :integral, :u, :p), NeuralPDE.var"#_RGF_ModTag", NeuralPDE.var"#_RGF_ModTag", (0x295b6845, 0xce1110b6, 0xa17beb73, 0xc449b733, 0xb3d6896e), Expr}, NeuralPDE.var"#12#13", NeuralPDE.var"#269#276"{NeuralPDE.var"#269#270#277"{typeof(NeuralPDE.numeric_derivative)}, Dict{Symbol, Int64}, Dict{Symbol, Int64}, GridTraining{Float64}}, typeof(NeuralPDE.numeric_derivative), Vector{NeuralPDE.Phi{Lux.Chain{NamedTuple{(:layer_1, :layer_2, :layer_3, :layer_4), Tuple{Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(identity), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}}}}, NamedTuple{(:layer_1, :layer_2, :layer_3, :layer_4), NTuple{4, NamedTuple{(), Tuple{}}}}}}, Vector{Float64}}, Matrix{Float64}}}, Vector{NeuralPDE.var"#74#75"{NeuralPDE.var"#209#210"{RuntimeGeneratedFunctions.RuntimeGeneratedFunction{(:cord, Symbol("##θ#293"), :phi, :derivative, :integral, :u, :p), NeuralPDE.var"#_RGF_ModTag", NeuralPDE.var"#_RGF_ModTag", (0xb5fb80bb, 0x61a62589, 0x2ec55669, 0x07bea97e, 0xb7ae3a52), Expr}, NeuralPDE.var"#12#13", NeuralPDE.var"#269#276"{NeuralPDE.var"#269#270#277"{typeof(NeuralPDE.numeric_derivative)}, Dict{Symbol, Int64}, Dict{Symbol, Int64}, GridTraining{Float64}}, typeof(NeuralPDE.numeric_derivative), Vector{NeuralPDE.Phi{Lux.Chain{NamedTuple{(:layer_1, :layer_2, :layer_3, :layer_4), Tuple{Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(identity), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}}}}, NamedTuple{(:layer_1, :layer_2, :layer_3, :layer_4), NTuple{4, NamedTuple{(), Tuple{}}}}}}, Vector{Float64}}, Matrix{Float64}}}, NeuralPDE.PINNRepresentation, Bool, Vector{Int64}, Int64, Vector{NeuralPDE.Phi{Lux.Chain{NamedTuple{(:layer_1, :layer_2, :layer_3, :layer_4), Tuple{Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(identity), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}}}}, NamedTuple{(:layer_1, :layer_2, :layer_3, :layer_4), NTuple{4, NamedTuple{(), Tuple{}}}}}}, Nothing, Bool, Vector{Float64}}, ::ComponentArrays.ComponentVector{Float64, Vector{Float64}, Tuple{ComponentArrays.Axis{(depvar = ViewAxis(1:177, Axis(T_in = ViewAxis(1:177, Axis(layer_1 = ViewAxis(1:24, Axis(weight = ViewAxis(1:16, ShapedAxis((8, 2), NamedTuple())), bias = ViewAxis(17:24, ShapedAxis((8, 1), NamedTuple())))), layer_2 = ViewAxis(25:96, Axis(weight = ViewAxis(1:64, ShapedAxis((8, 8), NamedTuple())), bias = ViewAxis(65:72, ShapedAxis((8, 1), NamedTuple())))), layer_3 = ViewAxis(97:168, Axis(weight = ViewAxis(1:64, ShapedAxis((8, 8), NamedTuple())), bias = ViewAxis(65:72, ShapedAxis((8, 1), NamedTuple())))), layer_4 = ViewAxis(169:177, Axis(weight = ViewAxis(1:8, ShapedAxis((1, 8), NamedTuple())), bias = ViewAxis(9:9, ShapedAxis((1, 1), NamedTuple())))))),)), p = 178:180)}}}, ::SciMLBase.NullParameters)
    @ Zygote C:\Users\loren\.julia\packages\Zygote\YYT6v\src\compiler\interface2.jl:0
 [45] _apply(::Function, ::Vararg{Any})
    @ Core .\boot.jl:838
 [46] adjoint
    @ C:\Users\loren\.julia\packages\Zygote\YYT6v\src\lib\lib.jl:203 [inlined]
 [47] _pullback
    @ C:\Users\loren\.julia\packages\ZygoteRules\4nXuu\src\adjoint.jl:66 [inlined]
 [48] _pullback
    @ C:\Users\loren\.julia\packages\SciMLBase\QqtZA\src\scimlfunctions.jl:3580 [inlined]
 [49] _pullback(::Zygote.Context{false}, ::OptimizationFunction{true, Optimization.AutoZygote, NeuralPDE.var"#full_loss_function#308"{NeuralPDE.var"#null_nonadaptive_loss#115", Vector{NeuralPDE.var"#74#75"{NeuralPDE.var"#209#210"{RuntimeGeneratedFunctions.RuntimeGeneratedFunction{(:cord, Symbol("##θ#293"), :phi, :derivative, :integral, :u, :p), NeuralPDE.var"#_RGF_ModTag", NeuralPDE.var"#_RGF_ModTag", (0x295b6845, 0xce1110b6, 0xa17beb73, 0xc449b733, 0xb3d6896e), Expr}, NeuralPDE.var"#12#13", NeuralPDE.var"#269#276"{NeuralPDE.var"#269#270#277"{typeof(NeuralPDE.numeric_derivative)}, Dict{Symbol, Int64}, Dict{Symbol, Int64}, GridTraining{Float64}}, typeof(NeuralPDE.numeric_derivative), Vector{NeuralPDE.Phi{Lux.Chain{NamedTuple{(:layer_1, :layer_2, :layer_3, :layer_4), Tuple{Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(identity), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}}}}, NamedTuple{(:layer_1, :layer_2, :layer_3, :layer_4), NTuple{4, NamedTuple{(), Tuple{}}}}}}, Vector{Float64}}, Matrix{Float64}}}, Vector{NeuralPDE.var"#74#75"{NeuralPDE.var"#209#210"{RuntimeGeneratedFunctions.RuntimeGeneratedFunction{(:cord, Symbol("##θ#293"), :phi, :derivative, :integral, :u, :p), NeuralPDE.var"#_RGF_ModTag", NeuralPDE.var"#_RGF_ModTag", (0xb5fb80bb, 0x61a62589, 0x2ec55669, 0x07bea97e, 0xb7ae3a52), Expr}, NeuralPDE.var"#12#13", NeuralPDE.var"#269#276"{NeuralPDE.var"#269#270#277"{typeof(NeuralPDE.numeric_derivative)}, Dict{Symbol, Int64}, Dict{Symbol, Int64}, GridTraining{Float64}}, typeof(NeuralPDE.numeric_derivative), Vector{NeuralPDE.Phi{Lux.Chain{NamedTuple{(:layer_1, :layer_2, :layer_3, :layer_4), Tuple{Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(identity), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}}}}, NamedTuple{(:layer_1, :layer_2, :layer_3, :layer_4), NTuple{4, NamedTuple{(), Tuple{}}}}}}, Vector{Float64}}, Matrix{Float64}}}, NeuralPDE.PINNRepresentation, Bool, Vector{Int64}, Int64, Vector{NeuralPDE.Phi{Lux.Chain{NamedTuple{(:layer_1, :layer_2, :layer_3, :layer_4), Tuple{Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(identity), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}}}}, NamedTuple{(:layer_1, :layer_2, :layer_3, :layer_4), NTuple{4, NamedTuple{(), Tuple{}}}}}}, Nothing, Bool, Vector{Float64}}, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, typeof(SciMLBase.DEFAULT_OBSERVED_NO_TIME), Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing}, ::ComponentArrays.ComponentVector{Float64, Vector{Float64}, Tuple{ComponentArrays.Axis{(depvar = ViewAxis(1:177, Axis(T_in = ViewAxis(1:177, Axis(layer_1 = ViewAxis(1:24, Axis(weight = ViewAxis(1:16, ShapedAxis((8, 2), NamedTuple())), bias = ViewAxis(17:24, ShapedAxis((8, 1), NamedTuple())))), layer_2 = ViewAxis(25:96, Axis(weight = ViewAxis(1:64, ShapedAxis((8, 8), NamedTuple())), bias = ViewAxis(65:72, ShapedAxis((8, 1), NamedTuple())))), layer_3 = ViewAxis(97:168, Axis(weight = ViewAxis(1:64, ShapedAxis((8, 8), NamedTuple())), bias = ViewAxis(65:72, ShapedAxis((8, 1), NamedTuple())))), layer_4 = ViewAxis(169:177, Axis(weight = ViewAxis(1:8, ShapedAxis((1, 8), NamedTuple())), bias = ViewAxis(9:9, ShapedAxis((1, 1), NamedTuple())))))),)), p = 178:180)}}}, ::SciMLBase.NullParameters)
    @ Zygote C:\Users\loren\.julia\packages\Zygote\YYT6v\src\compiler\interface2.jl:0
 [50] _apply(::Function, ::Vararg{Any})
    @ Core .\boot.jl:838
 [51] adjoint
    @ C:\Users\loren\.julia\packages\Zygote\YYT6v\src\lib\lib.jl:203 [inlined]
 [52] _pullback
    @ C:\Users\loren\.julia\packages\ZygoteRules\4nXuu\src\adjoint.jl:66 [inlined]
 [53] _pullback
    @ C:\Users\loren\.julia\packages\Optimization\vFala\src\function\zygote.jl:72 [inlined]
 [54] _pullback(ctx::Zygote.Context{false}, f::Optimization.var"#227#236"{OptimizationFunction{true, Optimization.AutoZygote, NeuralPDE.var"#full_loss_function#308"{NeuralPDE.var"#null_nonadaptive_loss#115", Vector{NeuralPDE.var"#74#75"{NeuralPDE.var"#209#210"{RuntimeGeneratedFunctions.RuntimeGeneratedFunction{(:cord, Symbol("##θ#293"), :phi, :derivative, :integral, :u, :p), NeuralPDE.var"#_RGF_ModTag", NeuralPDE.var"#_RGF_ModTag", (0x295b6845, 0xce1110b6, 0xa17beb73, 0xc449b733, 0xb3d6896e), Expr}, NeuralPDE.var"#12#13", NeuralPDE.var"#269#276"{NeuralPDE.var"#269#270#277"{typeof(NeuralPDE.numeric_derivative)}, Dict{Symbol, Int64}, Dict{Symbol, Int64}, GridTraining{Float64}}, typeof(NeuralPDE.numeric_derivative), Vector{NeuralPDE.Phi{Lux.Chain{NamedTuple{(:layer_1, :layer_2, :layer_3, :layer_4), Tuple{Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(identity), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}}}}, NamedTuple{(:layer_1, :layer_2, :layer_3, :layer_4), NTuple{4, NamedTuple{(), Tuple{}}}}}}, Vector{Float64}}, Matrix{Float64}}}, Vector{NeuralPDE.var"#74#75"{NeuralPDE.var"#209#210"{RuntimeGeneratedFunctions.RuntimeGeneratedFunction{(:cord, Symbol("##θ#293"), :phi, :derivative, :integral, :u, :p), NeuralPDE.var"#_RGF_ModTag", NeuralPDE.var"#_RGF_ModTag", (0xb5fb80bb, 0x61a62589, 0x2ec55669, 0x07bea97e, 0xb7ae3a52), Expr}, NeuralPDE.var"#12#13", NeuralPDE.var"#269#276"{NeuralPDE.var"#269#270#277"{typeof(NeuralPDE.numeric_derivative)}, Dict{Symbol, Int64}, Dict{Symbol, Int64}, GridTraining{Float64}}, typeof(NeuralPDE.numeric_derivative), Vector{NeuralPDE.Phi{Lux.Chain{NamedTuple{(:layer_1, :layer_2, :layer_3, :layer_4), Tuple{Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(identity), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}}}}, NamedTuple{(:layer_1, :layer_2, :layer_3, :layer_4), NTuple{4, NamedTuple{(), Tuple{}}}}}}, Vector{Float64}}, Matrix{Float64}}}, NeuralPDE.PINNRepresentation, Bool, Vector{Int64}, Int64, Vector{NeuralPDE.Phi{Lux.Chain{NamedTuple{(:layer_1, :layer_2, :layer_3, :layer_4), Tuple{Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(identity), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}}}}, NamedTuple{(:layer_1, :layer_2, :layer_3, :layer_4), NTuple{4, NamedTuple{(), Tuple{}}}}}}, Nothing, Bool, Vector{Float64}}, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, typeof(SciMLBase.DEFAULT_OBSERVED_NO_TIME), Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing}, Optimization.ReInitCache{ComponentArrays.ComponentVector{Float64, Vector{Float64}, Tuple{ComponentArrays.Axis{(depvar = ViewAxis(1:177, Axis(T_in = ViewAxis(1:177, Axis(layer_1 = ViewAxis(1:24, Axis(weight = ViewAxis(1:16, ShapedAxis((8, 2), NamedTuple())), bias = ViewAxis(17:24, ShapedAxis((8, 1), NamedTuple())))), layer_2 = ViewAxis(25:96, Axis(weight = ViewAxis(1:64, ShapedAxis((8, 8), NamedTuple())), bias = ViewAxis(65:72, ShapedAxis((8, 1), NamedTuple())))), layer_3 = ViewAxis(97:168, Axis(weight = ViewAxis(1:64, ShapedAxis((8, 8), NamedTuple())), bias = ViewAxis(65:72, ShapedAxis((8, 1), NamedTuple())))), layer_4 = ViewAxis(169:177, Axis(weight = ViewAxis(1:8, ShapedAxis((1, 8), NamedTuple())), bias = ViewAxis(9:9, ShapedAxis((1, 1), NamedTuple())))))),)), p = 178:180)}}}, SciMLBase.NullParameters}}, args::ComponentArrays.ComponentVector{Float64, Vector{Float64}, Tuple{ComponentArrays.Axis{(depvar = ViewAxis(1:177, Axis(T_in = ViewAxis(1:177, Axis(layer_1 = ViewAxis(1:24, Axis(weight = ViewAxis(1:16, ShapedAxis((8, 2), NamedTuple())), bias = ViewAxis(17:24, ShapedAxis((8, 1), NamedTuple())))), layer_2 = ViewAxis(25:96, Axis(weight = ViewAxis(1:64, ShapedAxis((8, 8), NamedTuple())), bias = ViewAxis(65:72, ShapedAxis((8, 1), NamedTuple())))), layer_3 = ViewAxis(97:168, Axis(weight = ViewAxis(1:64, ShapedAxis((8, 8), NamedTuple())), bias = ViewAxis(65:72, ShapedAxis((8, 1), NamedTuple())))), layer_4 = ViewAxis(169:177, Axis(weight = ViewAxis(1:8, ShapedAxis((1, 8), NamedTuple())), bias = ViewAxis(9:9, ShapedAxis((1, 1), NamedTuple())))))),)), p = 178:180)}}})
    @ Zygote C:\Users\loren\.julia\packages\Zygote\YYT6v\src\compiler\interface2.jl:0
 [55] _apply(::Function, ::Vararg{Any})
    @ Core .\boot.jl:838
 [56] adjoint
    @ C:\Users\loren\.julia\packages\Zygote\YYT6v\src\lib\lib.jl:203 [inlined]
 [57] _pullback
    @ C:\Users\loren\.julia\packages\ZygoteRules\4nXuu\src\adjoint.jl:66 [inlined]
 [58] _pullback
    @ C:\Users\loren\.julia\packages\Optimization\vFala\src\function\zygote.jl:76 [inlined]
 [59] _pullback(ctx::Zygote.Context{false}, f::Optimization.var"#229#238"{Tuple{}, Optimization.var"#227#236"{OptimizationFunction{true, Optimization.AutoZygote, NeuralPDE.var"#full_loss_function#308"{NeuralPDE.var"#null_nonadaptive_loss#115", Vector{NeuralPDE.var"#74#75"{NeuralPDE.var"#209#210"{RuntimeGeneratedFunctions.RuntimeGeneratedFunction{(:cord, Symbol("##θ#293"), :phi, :derivative, :integral, :u, :p), NeuralPDE.var"#_RGF_ModTag", NeuralPDE.var"#_RGF_ModTag", (0x295b6845, 0xce1110b6, 0xa17beb73, 0xc449b733, 0xb3d6896e), Expr}, NeuralPDE.var"#12#13", NeuralPDE.var"#269#276"{NeuralPDE.var"#269#270#277"{typeof(NeuralPDE.numeric_derivative)}, Dict{Symbol, Int64}, Dict{Symbol, Int64}, GridTraining{Float64}}, typeof(NeuralPDE.numeric_derivative), Vector{NeuralPDE.Phi{Lux.Chain{NamedTuple{(:layer_1, :layer_2, :layer_3, :layer_4), Tuple{Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(identity), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}}}}, NamedTuple{(:layer_1, :layer_2, :layer_3, :layer_4), NTuple{4, NamedTuple{(), Tuple{}}}}}}, Vector{Float64}}, Matrix{Float64}}}, Vector{NeuralPDE.var"#74#75"{NeuralPDE.var"#209#210"{RuntimeGeneratedFunctions.RuntimeGeneratedFunction{(:cord, Symbol("##θ#293"), :phi, :derivative, :integral, :u, :p), NeuralPDE.var"#_RGF_ModTag", NeuralPDE.var"#_RGF_ModTag", (0xb5fb80bb, 0x61a62589, 0x2ec55669, 0x07bea97e, 0xb7ae3a52), Expr}, NeuralPDE.var"#12#13", NeuralPDE.var"#269#276"{NeuralPDE.var"#269#270#277"{typeof(NeuralPDE.numeric_derivative)}, Dict{Symbol, Int64}, Dict{Symbol, Int64}, GridTraining{Float64}}, typeof(NeuralPDE.numeric_derivative), Vector{NeuralPDE.Phi{Lux.Chain{NamedTuple{(:layer_1, :layer_2, :layer_3, :layer_4), Tuple{Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(identity), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}}}}, NamedTuple{(:layer_1, :layer_2, :layer_3, :layer_4), NTuple{4, NamedTuple{(), Tuple{}}}}}}, Vector{Float64}}, Matrix{Float64}}}, NeuralPDE.PINNRepresentation, Bool, Vector{Int64}, Int64, Vector{NeuralPDE.Phi{Lux.Chain{NamedTuple{(:layer_1, :layer_2, :layer_3, :layer_4), Tuple{Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(identity), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}}}}, NamedTuple{(:layer_1, :layer_2, :layer_3, :layer_4), NTuple{4, NamedTuple{(), Tuple{}}}}}}, Nothing, Bool, Vector{Float64}}, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, typeof(SciMLBase.DEFAULT_OBSERVED_NO_TIME), Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing}, Optimization.ReInitCache{ComponentArrays.ComponentVector{Float64, Vector{Float64}, Tuple{ComponentArrays.Axis{(depvar = ViewAxis(1:177, Axis(T_in = ViewAxis(1:177, Axis(layer_1 = ViewAxis(1:24, Axis(weight = ViewAxis(1:16, ShapedAxis((8, 2), NamedTuple())), bias = ViewAxis(17:24, ShapedAxis((8, 1), NamedTuple())))), layer_2 = ViewAxis(25:96, Axis(weight = ViewAxis(1:64, ShapedAxis((8, 8), NamedTuple())), bias = ViewAxis(65:72, ShapedAxis((8, 1), NamedTuple())))), layer_3 = ViewAxis(97:168, Axis(weight = ViewAxis(1:64, ShapedAxis((8, 8), NamedTuple())), bias = ViewAxis(65:72, ShapedAxis((8, 1), NamedTuple())))), layer_4 = ViewAxis(169:177, Axis(weight = ViewAxis(1:8, ShapedAxis((1, 8), NamedTuple())), bias = ViewAxis(9:9, ShapedAxis((1, 1), NamedTuple())))))),)), p = 178:180)}}}, SciMLBase.NullParameters}}}, args::ComponentArrays.ComponentVector{Float64, Vector{Float64}, Tuple{ComponentArrays.Axis{(depvar = ViewAxis(1:177, Axis(T_in = ViewAxis(1:177, Axis(layer_1 = ViewAxis(1:24, Axis(weight = ViewAxis(1:16, ShapedAxis((8, 2), NamedTuple())), bias = ViewAxis(17:24, ShapedAxis((8, 1), NamedTuple())))), layer_2 = ViewAxis(25:96, Axis(weight = ViewAxis(1:64, ShapedAxis((8, 8), NamedTuple())), bias = ViewAxis(65:72, ShapedAxis((8, 1), NamedTuple())))), layer_3 = ViewAxis(97:168, Axis(weight = ViewAxis(1:64, ShapedAxis((8, 8), NamedTuple())), bias = ViewAxis(65:72, ShapedAxis((8, 1), NamedTuple())))), layer_4 = ViewAxis(169:177, Axis(weight = ViewAxis(1:8, ShapedAxis((1, 8), NamedTuple())), bias = ViewAxis(9:9, ShapedAxis((1, 1), NamedTuple())))))),)), p = 178:180)}}})
    @ Zygote C:\Users\loren\.julia\packages\Zygote\YYT6v\src\compiler\interface2.jl:0
 [60] pullback(f::Function, cx::Zygote.Context{false}, args::ComponentArrays.ComponentVector{Float64, Vector{Float64}, Tuple{ComponentArrays.Axis{(depvar = ViewAxis(1:177, Axis(T_in = ViewAxis(1:177, Axis(layer_1 = ViewAxis(1:24, Axis(weight = ViewAxis(1:16, ShapedAxis((8, 2), NamedTuple())), bias = ViewAxis(17:24, ShapedAxis((8, 1), NamedTuple())))), layer_2 = ViewAxis(25:96, Axis(weight = ViewAxis(1:64, ShapedAxis((8, 8), NamedTuple())), bias = ViewAxis(65:72, ShapedAxis((8, 1), NamedTuple())))), layer_3 = ViewAxis(97:168, Axis(weight = ViewAxis(1:64, ShapedAxis((8, 8), NamedTuple())), bias = ViewAxis(65:72, ShapedAxis((8, 1), NamedTuple())))), layer_4 = ViewAxis(169:177, Axis(weight = ViewAxis(1:8, ShapedAxis((1, 8), NamedTuple())), bias = ViewAxis(9:9, ShapedAxis((1, 1), NamedTuple())))))),)), p = 178:180)}}})
    @ Zygote C:\Users\loren\.julia\packages\Zygote\YYT6v\src\compiler\interface.jl:44
 [61] pullback
    @ C:\Users\loren\.julia\packages\Zygote\YYT6v\src\compiler\interface.jl:42 [inlined]
 [62] gradient(f::Function, args::ComponentArrays.ComponentVector{Float64, Vector{Float64}, Tuple{ComponentArrays.Axis{(depvar = ViewAxis(1:177, Axis(T_in = ViewAxis(1:177, Axis(layer_1 = ViewAxis(1:24, Axis(weight = ViewAxis(1:16, ShapedAxis((8, 2), NamedTuple())), bias = ViewAxis(17:24, ShapedAxis((8, 1), NamedTuple())))), layer_2 = ViewAxis(25:96, Axis(weight = ViewAxis(1:64, ShapedAxis((8, 8), NamedTuple())), bias = ViewAxis(65:72, ShapedAxis((8, 1), NamedTuple())))), layer_3 = ViewAxis(97:168, Axis(weight = ViewAxis(1:64, ShapedAxis((8, 8), NamedTuple())), bias = ViewAxis(65:72, ShapedAxis((8, 1), NamedTuple())))), layer_4 = ViewAxis(169:177, Axis(weight = ViewAxis(1:8, ShapedAxis((1, 8), NamedTuple())), bias = ViewAxis(9:9, ShapedAxis((1, 1), NamedTuple())))))),)), p = 178:180)}}})
    @ Zygote C:\Users\loren\.julia\packages\Zygote\YYT6v\src\compiler\interface.jl:96
 [63] (::Optimization.var"#228#237"{Optimization.var"#227#236"{OptimizationFunction{true, Optimization.AutoZygote, NeuralPDE.var"#full_loss_function#308"{NeuralPDE.var"#null_nonadaptive_loss#115", Vector{NeuralPDE.var"#74#75"{NeuralPDE.var"#209#210"{RuntimeGeneratedFunctions.RuntimeGeneratedFunction{(:cord, Symbol("##θ#293"), :phi, :derivative, :integral, :u, :p), NeuralPDE.var"#_RGF_ModTag", NeuralPDE.var"#_RGF_ModTag", (0x295b6845, 0xce1110b6, 0xa17beb73, 0xc449b733, 0xb3d6896e), Expr}, NeuralPDE.var"#12#13", NeuralPDE.var"#269#276"{NeuralPDE.var"#269#270#277"{typeof(NeuralPDE.numeric_derivative)}, Dict{Symbol, Int64}, Dict{Symbol, Int64}, GridTraining{Float64}}, typeof(NeuralPDE.numeric_derivative), Vector{NeuralPDE.Phi{Lux.Chain{NamedTuple{(:layer_1, :layer_2, :layer_3, :layer_4), Tuple{Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(identity), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}}}}, NamedTuple{(:layer_1, :layer_2, :layer_3, :layer_4), NTuple{4, NamedTuple{(), Tuple{}}}}}}, Vector{Float64}}, Matrix{Float64}}}, Vector{NeuralPDE.var"#74#75"{NeuralPDE.var"#209#210"{RuntimeGeneratedFunctions.RuntimeGeneratedFunction{(:cord, Symbol("##θ#293"), :phi, :derivative, :integral, :u, :p), NeuralPDE.var"#_RGF_ModTag", NeuralPDE.var"#_RGF_ModTag", (0xb5fb80bb, 0x61a62589, 0x2ec55669, 0x07bea97e, 0xb7ae3a52), Expr}, NeuralPDE.var"#12#13", NeuralPDE.var"#269#276"{NeuralPDE.var"#269#270#277"{typeof(NeuralPDE.numeric_derivative)}, Dict{Symbol, Int64}, Dict{Symbol, Int64}, GridTraining{Float64}}, typeof(NeuralPDE.numeric_derivative), Vector{NeuralPDE.Phi{Lux.Chain{NamedTuple{(:layer_1, :layer_2, :layer_3, :layer_4), Tuple{Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(identity), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}}}}, NamedTuple{(:layer_1, :layer_2, :layer_3, :layer_4), NTuple{4, NamedTuple{(), Tuple{}}}}}}, Vector{Float64}}, Matrix{Float64}}}, NeuralPDE.PINNRepresentation, Bool, Vector{Int64}, Int64, Vector{NeuralPDE.Phi{Lux.Chain{NamedTuple{(:layer_1, :layer_2, :layer_3, :layer_4), Tuple{Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(identity), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}}}}, NamedTuple{(:layer_1, :layer_2, :layer_3, :layer_4), NTuple{4, NamedTuple{(), Tuple{}}}}}}, Nothing, Bool, Vector{Float64}}, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, typeof(SciMLBase.DEFAULT_OBSERVED_NO_TIME), Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing}, Optimization.ReInitCache{ComponentArrays.ComponentVector{Float64, Vector{Float64}, Tuple{ComponentArrays.Axis{(depvar = ViewAxis(1:177, Axis(T_in = ViewAxis(1:177, Axis(layer_1 = ViewAxis(1:24, Axis(weight = ViewAxis(1:16, ShapedAxis((8, 2), NamedTuple())), bias = ViewAxis(17:24, ShapedAxis((8, 1), NamedTuple())))), layer_2 = ViewAxis(25:96, Axis(weight = ViewAxis(1:64, ShapedAxis((8, 8), NamedTuple())), bias = ViewAxis(65:72, ShapedAxis((8, 1), NamedTuple())))), layer_3 = ViewAxis(97:168, Axis(weight = ViewAxis(1:64, ShapedAxis((8, 8), NamedTuple())), bias = ViewAxis(65:72, ShapedAxis((8, 1), NamedTuple())))), layer_4 = ViewAxis(169:177, Axis(weight = ViewAxis(1:8, ShapedAxis((1, 8), NamedTuple())), bias = ViewAxis(9:9, ShapedAxis((1, 1), NamedTuple())))))),)), p = 178:180)}}}, SciMLBase.NullParameters}}})(::ComponentArrays.ComponentVector{Float64, Vector{Float64}, Tuple{ComponentArrays.Axis{(depvar = ViewAxis(1:177, Axis(T_in = ViewAxis(1:177, Axis(layer_1 = ViewAxis(1:24, Axis(weight = ViewAxis(1:16, ShapedAxis((8, 2), NamedTuple())), bias = ViewAxis(17:24, ShapedAxis((8, 1), NamedTuple())))), layer_2 = ViewAxis(25:96, Axis(weight = ViewAxis(1:64, ShapedAxis((8, 8), NamedTuple())), bias = ViewAxis(65:72, ShapedAxis((8, 1), NamedTuple())))), layer_3 = ViewAxis(97:168, Axis(weight = ViewAxis(1:64, ShapedAxis((8, 8), NamedTuple())), bias = ViewAxis(65:72, ShapedAxis((8, 1), NamedTuple())))), layer_4 = ViewAxis(169:177, Axis(weight = ViewAxis(1:8, ShapedAxis((1, 8), NamedTuple())), bias = ViewAxis(9:9, ShapedAxis((1, 1), NamedTuple())))))),)), p = 178:180)}}}, ::ComponentArrays.ComponentVector{Float64, Vector{Float64}, Tuple{ComponentArrays.Axis{(depvar = ViewAxis(1:177, Axis(T_in = ViewAxis(1:177, Axis(layer_1 = ViewAxis(1:24, Axis(weight = ViewAxis(1:16, ShapedAxis((8, 2), NamedTuple())), bias = ViewAxis(17:24, ShapedAxis((8, 1), NamedTuple())))), layer_2 = ViewAxis(25:96, Axis(weight = ViewAxis(1:64, ShapedAxis((8, 8), NamedTuple())), bias = ViewAxis(65:72, ShapedAxis((8, 1), NamedTuple())))), layer_3 = ViewAxis(97:168, Axis(weight = ViewAxis(1:64, ShapedAxis((8, 8), NamedTuple())), bias = ViewAxis(65:72, ShapedAxis((8, 1), NamedTuple())))), layer_4 = ViewAxis(169:177, Axis(weight = ViewAxis(1:8, ShapedAxis((1, 8), NamedTuple())), bias = ViewAxis(9:9, ShapedAxis((1, 1), NamedTuple())))))),)), p = 178:180)}}})
    @ Optimization C:\Users\loren\.julia\packages\Optimization\vFala\src\function\zygote.jl:74
 [64] (::OptimizationOptimJL.var"#10#16"{OptimizationOptimJL.OptimJLOptimizationCache{OptimizationFunction{false, Optimization.AutoZygote, OptimizationFunction{true, Optimization.AutoZygote, NeuralPDE.var"#full_loss_function#308"{NeuralPDE.var"#null_nonadaptive_loss#115", Vector{NeuralPDE.var"#74#75"{NeuralPDE.var"#209#210"{RuntimeGeneratedFunctions.RuntimeGeneratedFunction{(:cord, Symbol("##θ#293"), :phi, :derivative, :integral, :u, :p), NeuralPDE.var"#_RGF_ModTag", NeuralPDE.var"#_RGF_ModTag", (0x295b6845, 0xce1110b6, 0xa17beb73, 0xc449b733, 0xb3d6896e), Expr}, NeuralPDE.var"#12#13", NeuralPDE.var"#269#276"{NeuralPDE.var"#269#270#277"{typeof(NeuralPDE.numeric_derivative)}, Dict{Symbol, Int64}, Dict{Symbol, Int64}, GridTraining{Float64}}, typeof(NeuralPDE.numeric_derivative), Vector{NeuralPDE.Phi{Lux.Chain{NamedTuple{(:layer_1, :layer_2, :layer_3, :layer_4), Tuple{Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(identity), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}}}}, NamedTuple{(:layer_1, :layer_2, :layer_3, :layer_4), NTuple{4, NamedTuple{(), Tuple{}}}}}}, Vector{Float64}}, Matrix{Float64}}}, Vector{NeuralPDE.var"#74#75"{NeuralPDE.var"#209#210"{RuntimeGeneratedFunctions.RuntimeGeneratedFunction{(:cord, Symbol("##θ#293"), :phi, :derivative, :integral, :u, :p), NeuralPDE.var"#_RGF_ModTag", NeuralPDE.var"#_RGF_ModTag", (0xb5fb80bb, 0x61a62589, 0x2ec55669, 0x07bea97e, 0xb7ae3a52), Expr}, NeuralPDE.var"#12#13", NeuralPDE.var"#269#276"{NeuralPDE.var"#269#270#277"{typeof(NeuralPDE.numeric_derivative)}, Dict{Symbol, Int64}, Dict{Symbol, Int64}, GridTraining{Float64}}, typeof(NeuralPDE.numeric_derivative), Vector{NeuralPDE.Phi{Lux.Chain{NamedTuple{(:layer_1, :layer_2, :layer_3, :layer_4), Tuple{Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(identity), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}}}}, NamedTuple{(:layer_1, :layer_2, :layer_3, :layer_4), NTuple{4, NamedTuple{(), Tuple{}}}}}}, Vector{Float64}}, Matrix{Float64}}}, NeuralPDE.PINNRepresentation, Bool, Vector{Int64}, Int64, Vector{NeuralPDE.Phi{Lux.Chain{NamedTuple{(:layer_1, :layer_2, :layer_3, :layer_4), Tuple{Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(identity), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}}}}, NamedTuple{(:layer_1, :layer_2, :layer_3, :layer_4), NTuple{4, NamedTuple{(), Tuple{}}}}}}, Nothing, Bool, Vector{Float64}}, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, typeof(SciMLBase.DEFAULT_OBSERVED_NO_TIME), Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing}, Optimization.var"#228#237"{Optimization.var"#227#236"{OptimizationFunction{true, Optimization.AutoZygote, NeuralPDE.var"#full_loss_function#308"{NeuralPDE.var"#null_nonadaptive_loss#115", Vector{NeuralPDE.var"#74#75"{NeuralPDE.var"#209#210"{RuntimeGeneratedFunctions.RuntimeGeneratedFunction{(:cord, Symbol("##θ#293"), :phi, :derivative, :integral, :u, :p), NeuralPDE.var"#_RGF_ModTag", NeuralPDE.var"#_RGF_ModTag", (0x295b6845, 0xce1110b6, 0xa17beb73, 0xc449b733, 0xb3d6896e), Expr}, NeuralPDE.var"#12#13", NeuralPDE.var"#269#276"{NeuralPDE.var"#269#270#277"{typeof(NeuralPDE.numeric_derivative)}, Dict{Symbol, Int64}, Dict{Symbol, Int64}, GridTraining{Float64}}, typeof(NeuralPDE.numeric_derivative), Vector{NeuralPDE.Phi{Lux.Chain{NamedTuple{(:layer_1, :layer_2, :layer_3, :layer_4), Tuple{Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(identity), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}}}}, NamedTuple{(:layer_1, :layer_2, :layer_3, :layer_4), NTuple{4, NamedTuple{(), Tuple{}}}}}}, Vector{Float64}}, Matrix{Float64}}}, Vector{NeuralPDE.var"#74#75"{NeuralPDE.var"#209#210"{RuntimeGeneratedFunctions.RuntimeGeneratedFunction{(:cord, Symbol("##θ#293"), :phi, :derivative, :integral, :u, :p), NeuralPDE.var"#_RGF_ModTag", NeuralPDE.var"#_RGF_ModTag", (0xb5fb80bb, 0x61a62589, 0x2ec55669, 0x07bea97e, 0xb7ae3a52), Expr}, NeuralPDE.var"#12#13", NeuralPDE.var"#269#276"{NeuralPDE.var"#269#270#277"{typeof(NeuralPDE.numeric_derivative)}, Dict{Symbol, Int64}, Dict{Symbol, Int64}, GridTraining{Float64}}, typeof(NeuralPDE.numeric_derivative), Vector{NeuralPDE.Phi{Lux.Chain{NamedTuple{(:layer_1, :layer_2, :layer_3, :layer_4), Tuple{Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(identity), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}}}}, NamedTuple{(:layer_1, :layer_2, :layer_3, :layer_4), NTuple{4, NamedTuple{(), Tuple{}}}}}}, Vector{Float64}}, Matrix{Float64}}}, NeuralPDE.PINNRepresentation, Bool, Vector{Int64}, Int64, Vector{NeuralPDE.Phi{Lux.Chain{NamedTuple{(:layer_1, :layer_2, :layer_3, :layer_4), Tuple{Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(identity), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}}}}, NamedTuple{(:layer_1, :layer_2, :layer_3, :layer_4), NTuple{4, NamedTuple{(), Tuple{}}}}}}, Nothing, Bool, Vector{Float64}}, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, typeof(SciMLBase.DEFAULT_OBSERVED_NO_TIME), Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing}, Optimization.ReInitCache{ComponentArrays.ComponentVector{Float64, Vector{Float64}, Tuple{ComponentArrays.Axis{(depvar = ViewAxis(1:177, Axis(T_in = ViewAxis(1:177, Axis(layer_1 = ViewAxis(1:24, Axis(weight = ViewAxis(1:16, ShapedAxis((8, 2), NamedTuple())), bias = ViewAxis(17:24, ShapedAxis((8, 1), NamedTuple())))), layer_2 = ViewAxis(25:96, Axis(weight = ViewAxis(1:64, ShapedAxis((8, 8), NamedTuple())), bias = ViewAxis(65:72, ShapedAxis((8, 1), NamedTuple())))), layer_3 = ViewAxis(97:168, Axis(weight = ViewAxis(1:64, ShapedAxis((8, 8), NamedTuple())), bias = ViewAxis(65:72, ShapedAxis((8, 1), NamedTuple())))), layer_4 = ViewAxis(169:177, Axis(weight = ViewAxis(1:8, ShapedAxis((1, 8), NamedTuple())), bias = ViewAxis(9:9, ShapedAxis((1, 1), NamedTuple())))))),)), p = 178:180)}}}, SciMLBase.NullParameters}}}, Optimization.var"#231#240"{Optimization.var"#227#236"{OptimizationFunction{true, Optimization.AutoZygote, NeuralPDE.var"#full_loss_function#308"{NeuralPDE.var"#null_nonadaptive_loss#115", Vector{NeuralPDE.var"#74#75"{NeuralPDE.var"#209#210"{RuntimeGeneratedFunctions.RuntimeGeneratedFunction{(:cord, Symbol("##θ#293"), :phi, :derivative, :integral, :u, :p), NeuralPDE.var"#_RGF_ModTag", NeuralPDE.var"#_RGF_ModTag", (0x295b6845, 0xce1110b6, 0xa17beb73, 0xc449b733, 0xb3d6896e), Expr}, NeuralPDE.var"#12#13", NeuralPDE.var"#269#276"{NeuralPDE.var"#269#270#277"{typeof(NeuralPDE.numeric_derivative)}, Dict{Symbol, Int64}, Dict{Symbol, Int64}, GridTraining{Float64}}, typeof(NeuralPDE.numeric_derivative), Vector{NeuralPDE.Phi{Lux.Chain{NamedTuple{(:layer_1, :layer_2, :layer_3, :layer_4), Tuple{Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(identity), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}}}}, NamedTuple{(:layer_1, :layer_2, :layer_3, :layer_4), NTuple{4, NamedTuple{(), Tuple{}}}}}}, Vector{Float64}}, Matrix{Float64}}}, Vector{NeuralPDE.var"#74#75"{NeuralPDE.var"#209#210"{RuntimeGeneratedFunctions.RuntimeGeneratedFunction{(:cord, Symbol("##θ#293"), :phi, :derivative, :integral, :u, :p), NeuralPDE.var"#_RGF_ModTag", NeuralPDE.var"#_RGF_ModTag", (0xb5fb80bb, 0x61a62589, 0x2ec55669, 0x07bea97e, 0xb7ae3a52), Expr}, NeuralPDE.var"#12#13", NeuralPDE.var"#269#276"{NeuralPDE.var"#269#270#277"{typeof(NeuralPDE.numeric_derivative)}, Dict{Symbol, Int64}, Dict{Symbol, Int64}, GridTraining{Float64}}, typeof(NeuralPDE.numeric_derivative), Vector{NeuralPDE.Phi{Lux.Chain{NamedTuple{(:layer_1, :layer_2, :layer_3, :layer_4), Tuple{Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(identity), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}}}}, NamedTuple{(:layer_1, :layer_2, :layer_3, :layer_4), NTuple{4, NamedTuple{(), Tuple{}}}}}}, Vector{Float64}}, Matrix{Float64}}}, NeuralPDE.PINNRepresentation, Bool, Vector{Int64}, Int64, Vector{NeuralPDE.Phi{Lux.Chain{NamedTuple{(:layer_1, :layer_2, :layer_3, :layer_4), Tuple{Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(identity), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}}}}, NamedTuple{(:layer_1, :layer_2, :layer_3, :layer_4), NTuple{4, NamedTuple{(), Tuple{}}}}}}, Nothing, Bool, Vector{Float64}}, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, typeof(SciMLBase.DEFAULT_OBSERVED_NO_TIME), Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing}, Optimization.ReInitCache{ComponentArrays.ComponentVector{Float64, Vector{Float64}, Tuple{ComponentArrays.Axis{(depvar = ViewAxis(1:177, Axis(T_in = ViewAxis(1:177, Axis(layer_1 = ViewAxis(1:24, Axis(weight = ViewAxis(1:16, ShapedAxis((8, 2), NamedTuple())), bias = ViewAxis(17:24, ShapedAxis((8, 1), NamedTuple())))), layer_2 = ViewAxis(25:96, Axis(weight = ViewAxis(1:64, ShapedAxis((8, 8), NamedTuple())), bias = ViewAxis(65:72, ShapedAxis((8, 1), NamedTuple())))), layer_3 = ViewAxis(97:168, Axis(weight = ViewAxis(1:64, ShapedAxis((8, 8), NamedTuple())), bias = ViewAxis(65:72, ShapedAxis((8, 1), NamedTuple())))), layer_4 = ViewAxis(169:177, Axis(weight = ViewAxis(1:8, ShapedAxis((1, 8), NamedTuple())), bias = ViewAxis(9:9, ShapedAxis((1, 1), NamedTuple())))))),)), p = 178:180)}}}, SciMLBase.NullParameters}}}, Optimization.var"#235#244", Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, typeof(SciMLBase.DEFAULT_OBSERVED_NO_TIME), Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing}, Optimization.ReInitCache{ComponentArrays.ComponentVector{Float64, Vector{Float64}, Tuple{ComponentArrays.Axis{(depvar = ViewAxis(1:177, Axis(T_in = ViewAxis(1:177, Axis(layer_1 = ViewAxis(1:24, Axis(weight = ViewAxis(1:16, ShapedAxis((8, 2), NamedTuple())), bias = ViewAxis(17:24, ShapedAxis((8, 1), NamedTuple())))), layer_2 = ViewAxis(25:96, Axis(weight = ViewAxis(1:64, ShapedAxis((8, 8), NamedTuple())), bias = ViewAxis(65:72, ShapedAxis((8, 1), NamedTuple())))), layer_3 = ViewAxis(97:168, Axis(weight = ViewAxis(1:64, ShapedAxis((8, 8), NamedTuple())), bias = ViewAxis(65:72, ShapedAxis((8, 1), NamedTuple())))), layer_4 = ViewAxis(169:177, Axis(weight = ViewAxis(1:8, ShapedAxis((1, 8), NamedTuple())), bias = ViewAxis(9:9, ShapedAxis((1, 1), NamedTuple())))))),)), p = 178:180)}}}, SciMLBase.NullParameters}, Nothing, Nothing, Nothing, Nothing, Nothing, BFGS{LineSearches.InitialStatic{Float64}, LineSearches.HagerZhang{Float64, Base.RefValue{Bool}}, Nothing, Nothing, Flat}, Base.Iterators.Cycle{Tuple{Optimization.NullData}}, Bool, var"#9#10"}, OptimizationOptimJL.var"#9#15"{OptimizationOptimJL.OptimJLOptimizationCache{OptimizationFunction{false, Optimization.AutoZygote, OptimizationFunction{true, Optimization.AutoZygote, NeuralPDE.var"#full_loss_function#308"{NeuralPDE.var"#null_nonadaptive_loss#115", Vector{NeuralPDE.var"#74#75"{NeuralPDE.var"#209#210"{RuntimeGeneratedFunctions.RuntimeGeneratedFunction{(:cord, Symbol("##θ#293"), :phi, :derivative, :integral, :u, :p), NeuralPDE.var"#_RGF_ModTag", NeuralPDE.var"#_RGF_ModTag", (0x295b6845, 0xce1110b6, 0xa17beb73, 0xc449b733, 0xb3d6896e), Expr}, NeuralPDE.var"#12#13", NeuralPDE.var"#269#276"{NeuralPDE.var"#269#270#277"{typeof(NeuralPDE.numeric_derivative)}, Dict{Symbol, Int64}, Dict{Symbol, Int64}, GridTraining{Float64}}, typeof(NeuralPDE.numeric_derivative), Vector{NeuralPDE.Phi{Lux.Chain{NamedTuple{(:layer_1, :layer_2, :layer_3, :layer_4), Tuple{Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(identity), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}}}}, NamedTuple{(:layer_1, :layer_2, :layer_3, :layer_4), NTuple{4, NamedTuple{(), Tuple{}}}}}}, Vector{Float64}}, Matrix{Float64}}}, Vector{NeuralPDE.var"#74#75"{NeuralPDE.var"#209#210"{RuntimeGeneratedFunctions.RuntimeGeneratedFunction{(:cord, Symbol("##θ#293"), :phi, :derivative, :integral, :u, :p), NeuralPDE.var"#_RGF_ModTag", NeuralPDE.var"#_RGF_ModTag", (0xb5fb80bb, 0x61a62589, 0x2ec55669, 0x07bea97e, 0xb7ae3a52), Expr}, NeuralPDE.var"#12#13", NeuralPDE.var"#269#276"{NeuralPDE.var"#269#270#277"{typeof(NeuralPDE.numeric_derivative)}, Dict{Symbol, Int64}, Dict{Symbol, Int64}, GridTraining{Float64}}, typeof(NeuralPDE.numeric_derivative), Vector{NeuralPDE.Phi{Lux.Chain{NamedTuple{(:layer_1, :layer_2, :layer_3, :layer_4), Tuple{Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(identity), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}}}}, NamedTuple{(:layer_1, :layer_2, :layer_3, :layer_4), NTuple{4, NamedTuple{(), Tuple{}}}}}}, Vector{Float64}}, Matrix{Float64}}}, NeuralPDE.PINNRepresentation, Bool, Vector{Int64}, Int64, Vector{NeuralPDE.Phi{Lux.Chain{NamedTuple{(:layer_1, :layer_2, :layer_3, :layer_4), Tuple{Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(identity), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}}}}, NamedTuple{(:layer_1, :layer_2, :layer_3, :layer_4), NTuple{4, NamedTuple{(), Tuple{}}}}}}, Nothing, Bool, Vector{Float64}}, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, typeof(SciMLBase.DEFAULT_OBSERVED_NO_TIME), Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing}, Optimization.var"#228#237"{Optimization.var"#227#236"{OptimizationFunction{true, Optimization.AutoZygote, NeuralPDE.var"#full_loss_function#308"{NeuralPDE.var"#null_nonadaptive_loss#115", Vector{NeuralPDE.var"#74#75"{NeuralPDE.var"#209#210"{RuntimeGeneratedFunctions.RuntimeGeneratedFunction{(:cord, Symbol("##θ#293"), :phi, :derivative, :integral, :u, :p), NeuralPDE.var"#_RGF_ModTag", NeuralPDE.var"#_RGF_ModTag", (0x295b6845, 0xce1110b6, 0xa17beb73, 0xc449b733, 0xb3d6896e), Expr}, NeuralPDE.var"#12#13", NeuralPDE.var"#269#276"{NeuralPDE.var"#269#270#277"{typeof(NeuralPDE.numeric_derivative)}, Dict{Symbol, Int64}, Dict{Symbol, Int64}, GridTraining{Float64}}, typeof(NeuralPDE.numeric_derivative), Vector{NeuralPDE.Phi{Lux.Chain{NamedTuple{(:layer_1, :layer_2, :layer_3, :layer_4), Tuple{Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(identity), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}}}}, NamedTuple{(:layer_1, :layer_2, :layer_3, :layer_4), NTuple{4, NamedTuple{(), Tuple{}}}}}}, Vector{Float64}}, Matrix{Float64}}}, Vector{NeuralPDE.var"#74#75"{NeuralPDE.var"#209#210"{RuntimeGeneratedFunctions.RuntimeGeneratedFunction{(:cord, Symbol("##θ#293"), :phi, :derivative, :integral, :u, :p), NeuralPDE.var"#_RGF_ModTag", NeuralPDE.var"#_RGF_ModTag", (0xb5fb80bb, 0x61a62589, 0x2ec55669, 0x07bea97e, 0xb7ae3a52), Expr}, NeuralPDE.var"#12#13", NeuralPDE.var"#269#276"{NeuralPDE.var"#269#270#277"{typeof(NeuralPDE.numeric_derivative)}, Dict{Symbol, Int64}, Dict{Symbol, Int64}, GridTraining{Float64}}, typeof(NeuralPDE.numeric_derivative), Vector{NeuralPDE.Phi{Lux.Chain{NamedTuple{(:layer_1, :layer_2, :layer_3, :layer_4), Tuple{Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(identity), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}}}}, NamedTuple{(:layer_1, :layer_2, :layer_3, :layer_4), NTuple{4, NamedTuple{(), Tuple{}}}}}}, Vector{Float64}}, Matrix{Float64}}}, NeuralPDE.PINNRepresentation, Bool, Vector{Int64}, Int64, Vector{NeuralPDE.Phi{Lux.Chain{NamedTuple{(:layer_1, :layer_2, :layer_3, :layer_4), Tuple{Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(identity), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}}}}, NamedTuple{(:layer_1, :layer_2, :layer_3, :layer_4), NTuple{4, NamedTuple{(), Tuple{}}}}}}, Nothing, Bool, Vector{Float64}}, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, typeof(SciMLBase.DEFAULT_OBSERVED_NO_TIME), Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing}, Optimization.ReInitCache{ComponentArrays.ComponentVector{Float64, Vector{Float64}, Tuple{ComponentArrays.Axis{(depvar = ViewAxis(1:177, Axis(T_in = ViewAxis(1:177, Axis(layer_1 = ViewAxis(1:24, Axis(weight = ViewAxis(1:16, ShapedAxis((8, 2), NamedTuple())), bias = ViewAxis(17:24, ShapedAxis((8, 1), NamedTuple())))), layer_2 = ViewAxis(25:96, Axis(weight = ViewAxis(1:64, ShapedAxis((8, 8), NamedTuple())), bias = ViewAxis(65:72, ShapedAxis((8, 1), NamedTuple())))), layer_3 = ViewAxis(97:168, Axis(weight = ViewAxis(1:64, ShapedAxis((8, 8), NamedTuple())), bias = ViewAxis(65:72, ShapedAxis((8, 1), NamedTuple())))), layer_4 = ViewAxis(169:177, Axis(weight = ViewAxis(1:8, ShapedAxis((1, 8), NamedTuple())), bias = ViewAxis(9:9, ShapedAxis((1, 1), NamedTuple())))))),)), p = 178:180)}}}, SciMLBase.NullParameters}}}, Optimization.var"#231#240"{Optimization.var"#227#236"{OptimizationFunction{true, Optimization.AutoZygote, NeuralPDE.var"#full_loss_function#308"{NeuralPDE.var"#null_nonadaptive_loss#115", Vector{NeuralPDE.var"#74#75"{NeuralPDE.var"#209#210"{RuntimeGeneratedFunctions.RuntimeGeneratedFunction{(:cord, Symbol("##θ#293"), :phi, :derivative, :integral, :u, :p), NeuralPDE.var"#_RGF_ModTag", NeuralPDE.var"#_RGF_ModTag", (0x295b6845, 0xce1110b6, 0xa17beb73, 0xc449b733, 0xb3d6896e), Expr}, NeuralPDE.var"#12#13", NeuralPDE.var"#269#276"{NeuralPDE.var"#269#270#277"{typeof(NeuralPDE.numeric_derivative)}, Dict{Symbol, Int64}, Dict{Symbol, Int64}, GridTraining{Float64}}, typeof(NeuralPDE.numeric_derivative), Vector{NeuralPDE.Phi{Lux.Chain{NamedTuple{(:layer_1, :layer_2, :layer_3, :layer_4), Tuple{Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(identity), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}}}}, NamedTuple{(:layer_1, :layer_2, :layer_3, :layer_4), NTuple{4, NamedTuple{(), Tuple{}}}}}}, Vector{Float64}}, Matrix{Float64}}}, Vector{NeuralPDE.var"#74#75"{NeuralPDE.var"#209#210"{RuntimeGeneratedFunctions.RuntimeGeneratedFunction{(:cord, Symbol("##θ#293"), :phi, :derivative, :integral, :u, :p), NeuralPDE.var"#_RGF_ModTag", NeuralPDE.var"#_RGF_ModTag", (0xb5fb80bb, 0x61a62589, 0x2ec55669, 0x07bea97e, 0xb7ae3a52), Expr}, NeuralPDE.var"#12#13", NeuralPDE.var"#269#276"{NeuralPDE.var"#269#270#277"{typeof(NeuralPDE.numeric_derivative)}, Dict{Symbol, Int64}, Dict{Symbol, Int64}, GridTraining{Float64}}, typeof(NeuralPDE.numeric_derivative), Vector{NeuralPDE.Phi{Lux.Chain{NamedTuple{(:layer_1, :layer_2, :layer_3, :layer_4), Tuple{Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(identity), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}}}}, NamedTuple{(:layer_1, :layer_2, :layer_3, :layer_4), NTuple{4, NamedTuple{(), Tuple{}}}}}}, Vector{Float64}}, Matrix{Float64}}}, NeuralPDE.PINNRepresentation, Bool, Vector{Int64}, Int64, Vector{NeuralPDE.Phi{Lux.Chain{NamedTuple{(:layer_1, :layer_2, :layer_3, :layer_4), Tuple{Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(identity), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}}}}, NamedTuple{(:layer_1, :layer_2, :layer_3, :layer_4), NTuple{4, NamedTuple{(), Tuple{}}}}}}, Nothing, Bool, Vector{Float64}}, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, typeof(SciMLBase.DEFAULT_OBSERVED_NO_TIME), Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing}, Optimization.ReInitCache{ComponentArrays.ComponentVector{Float64, Vector{Float64}, Tuple{ComponentArrays.Axis{(depvar = ViewAxis(1:177, Axis(T_in = ViewAxis(1:177, Axis(layer_1 = ViewAxis(1:24, Axis(weight = ViewAxis(1:16, ShapedAxis((8, 2), NamedTuple())), bias = ViewAxis(17:24, ShapedAxis((8, 1), NamedTuple())))), layer_2 = ViewAxis(25:96, Axis(weight = ViewAxis(1:64, ShapedAxis((8, 8), NamedTuple())), bias = ViewAxis(65:72, ShapedAxis((8, 1), NamedTuple())))), layer_3 = ViewAxis(97:168, Axis(weight = ViewAxis(1:64, ShapedAxis((8, 8), NamedTuple())), bias = ViewAxis(65:72, ShapedAxis((8, 1), NamedTuple())))), layer_4 = ViewAxis(169:177, Axis(weight = ViewAxis(1:8, ShapedAxis((1, 8), NamedTuple())), bias = ViewAxis(9:9, ShapedAxis((1, 1), NamedTuple())))))),)), p = 178:180)}}}, SciMLBase.NullParameters}}}, Optimization.var"#235#244", Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, typeof(SciMLBase.DEFAULT_OBSERVED_NO_TIME), Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing}, Optimization.ReInitCache{ComponentArrays.ComponentVector{Float64, Vector{Float64}, Tuple{ComponentArrays.Axis{(depvar = ViewAxis(1:177, Axis(T_in = ViewAxis(1:177, Axis(layer_1 = ViewAxis(1:24, Axis(weight = ViewAxis(1:16, ShapedAxis((8, 2), NamedTuple())), bias = ViewAxis(17:24, ShapedAxis((8, 1), NamedTuple())))), layer_2 = ViewAxis(25:96, Axis(weight = ViewAxis(1:64, ShapedAxis((8, 8), NamedTuple())), bias = ViewAxis(65:72, ShapedAxis((8, 1), NamedTuple())))), layer_3 = ViewAxis(97:168, Axis(weight = ViewAxis(1:64, ShapedAxis((8, 8), NamedTuple())), bias = ViewAxis(65:72, ShapedAxis((8, 1), NamedTuple())))), layer_4 = ViewAxis(169:177, Axis(weight = ViewAxis(1:8, ShapedAxis((1, 8), NamedTuple())), bias = ViewAxis(9:9, ShapedAxis((1, 1), NamedTuple())))))),)), p = 178:180)}}}, SciMLBase.NullParameters}, Nothing, Nothing, Nothing, Nothing, Nothing, BFGS{LineSearches.InitialStatic{Float64}, LineSearches.HagerZhang{Float64, Base.RefValue{Bool}}, Nothing, Nothing, Flat}, Base.Iterators.Cycle{Tuple{Optimization.NullData}}, Bool, var"#9#10"}}})(G::ComponentArrays.ComponentVector{Float64, Vector{Float64}, Tuple{ComponentArrays.Axis{(depvar = ViewAxis(1:177, Axis(T_in = ViewAxis(1:177, Axis(layer_1 = ViewAxis(1:24, Axis(weight = ViewAxis(1:16, ShapedAxis((8, 2), NamedTuple())), bias = ViewAxis(17:24, ShapedAxis((8, 1), NamedTuple())))), layer_2 = ViewAxis(25:96, Axis(weight = ViewAxis(1:64, ShapedAxis((8, 8), NamedTuple())), bias = ViewAxis(65:72, ShapedAxis((8, 1), NamedTuple())))), layer_3 = ViewAxis(97:168, Axis(weight = ViewAxis(1:64, ShapedAxis((8, 8), NamedTuple())), bias = ViewAxis(65:72, ShapedAxis((8, 1), NamedTuple())))), layer_4 = ViewAxis(169:177, Axis(weight = ViewAxis(1:8, ShapedAxis((1, 8), NamedTuple())), bias = ViewAxis(9:9, ShapedAxis((1, 1), NamedTuple())))))),)), p = 178:180)}}}, θ::ComponentArrays.ComponentVector{Float64, Vector{Float64}, Tuple{ComponentArrays.Axis{(depvar = ViewAxis(1:177, Axis(T_in = ViewAxis(1:177, Axis(layer_1 = ViewAxis(1:24, Axis(weight = ViewAxis(1:16, ShapedAxis((8, 2), NamedTuple())), bias = ViewAxis(17:24, ShapedAxis((8, 1), NamedTuple())))), layer_2 = ViewAxis(25:96, Axis(weight = ViewAxis(1:64, ShapedAxis((8, 8), NamedTuple())), bias = ViewAxis(65:72, ShapedAxis((8, 1), NamedTuple())))), layer_3 = ViewAxis(97:168, Axis(weight = ViewAxis(1:64, ShapedAxis((8, 8), NamedTuple())), bias = ViewAxis(65:72, ShapedAxis((8, 1), NamedTuple())))), layer_4 = ViewAxis(169:177, Axis(weight = ViewAxis(1:8, ShapedAxis((1, 8), NamedTuple())), bias = ViewAxis(9:9, ShapedAxis((1, 1), NamedTuple())))))),)), p = 178:180)}}})
    @ OptimizationOptimJL C:\Users\loren\.julia\packages\OptimizationOptimJL\uRfW9\src\OptimizationOptimJL.jl:200
 [65] value_gradient!!(obj::TwiceDifferentiable{Float64, ComponentArrays.ComponentVector{Float64, Vector{Float64}, Tuple{ComponentArrays.Axis{(depvar = ViewAxis(1:177, Axis(T_in = ViewAxis(1:177, Axis(layer_1 = ViewAxis(1:24, Axis(weight = ViewAxis(1:16, ShapedAxis((8, 2), NamedTuple())), bias = ViewAxis(17:24, ShapedAxis((8, 1), NamedTuple())))), layer_2 = ViewAxis(25:96, Axis(weight = ViewAxis(1:64, ShapedAxis((8, 8), NamedTuple())), bias = ViewAxis(65:72, ShapedAxis((8, 1), NamedTuple())))), layer_3 = ViewAxis(97:168, Axis(weight = ViewAxis(1:64, ShapedAxis((8, 8), NamedTuple())), bias = ViewAxis(65:72, ShapedAxis((8, 1), NamedTuple())))), layer_4 = ViewAxis(169:177, Axis(weight = ViewAxis(1:8, ShapedAxis((1, 8), NamedTuple())), bias = ViewAxis(9:9, ShapedAxis((1, 1), NamedTuple())))))),)), p = 178:180)}}}, ComponentArrays.ComponentMatrix{Float64, Matrix{Float64}, Tuple{ComponentArrays.Axis{(depvar = ViewAxis(1:177, Axis(T_in = ViewAxis(1:177, Axis(layer_1 = ViewAxis(1:24, Axis(weight = ViewAxis(1:16, ShapedAxis((8, 2), NamedTuple())), bias = ViewAxis(17:24, ShapedAxis((8, 1), NamedTuple())))), layer_2 = ViewAxis(25:96, Axis(weight = ViewAxis(1:64, ShapedAxis((8, 8), NamedTuple())), bias = ViewAxis(65:72, ShapedAxis((8, 1), NamedTuple())))), layer_3 = ViewAxis(97:168, Axis(weight = ViewAxis(1:64, ShapedAxis((8, 8), NamedTuple())), bias = ViewAxis(65:72, ShapedAxis((8, 1), NamedTuple())))), layer_4 = ViewAxis(169:177, Axis(weight = ViewAxis(1:8, ShapedAxis((1, 8), NamedTuple())), bias = ViewAxis(9:9, ShapedAxis((1, 1), NamedTuple())))))),)), p = 178:180)}, ComponentArrays.Axis{(depvar = ViewAxis(1:177, Axis(T_in = ViewAxis(1:177, Axis(layer_1 = ViewAxis(1:24, Axis(weight = ViewAxis(1:16, ShapedAxis((8, 2), NamedTuple())), bias = ViewAxis(17:24, ShapedAxis((8, 1), NamedTuple())))), layer_2 = ViewAxis(25:96, Axis(weight = ViewAxis(1:64, ShapedAxis((8, 8), NamedTuple())), bias = ViewAxis(65:72, ShapedAxis((8, 1), NamedTuple())))), layer_3 = ViewAxis(97:168, Axis(weight = ViewAxis(1:64, ShapedAxis((8, 8), NamedTuple())), bias = ViewAxis(65:72, ShapedAxis((8, 1), NamedTuple())))), layer_4 = ViewAxis(169:177, Axis(weight = ViewAxis(1:8, ShapedAxis((1, 8), NamedTuple())), bias = ViewAxis(9:9, ShapedAxis((1, 1), NamedTuple())))))),)), p = 178:180)}}}, ComponentArrays.ComponentVector{Float64, Vector{Float64}, Tuple{ComponentArrays.Axis{(depvar = ViewAxis(1:177, Axis(T_in = ViewAxis(1:177, Axis(layer_1 = ViewAxis(1:24, Axis(weight = ViewAxis(1:16, ShapedAxis((8, 2), NamedTuple())), bias = ViewAxis(17:24, ShapedAxis((8, 1), NamedTuple())))), layer_2 = ViewAxis(25:96, Axis(weight = ViewAxis(1:64, ShapedAxis((8, 8), NamedTuple())), bias = ViewAxis(65:72, ShapedAxis((8, 1), NamedTuple())))), layer_3 = ViewAxis(97:168, Axis(weight = ViewAxis(1:64, ShapedAxis((8, 8), NamedTuple())), bias = ViewAxis(65:72, ShapedAxis((8, 1), NamedTuple())))), layer_4 = ViewAxis(169:177, Axis(weight = ViewAxis(1:8, ShapedAxis((1, 8), NamedTuple())), bias = ViewAxis(9:9, ShapedAxis((1, 1), NamedTuple())))))),)), p = 178:180)}}}}, x::ComponentArrays.ComponentVector{Float64, Vector{Float64}, Tuple{ComponentArrays.Axis{(depvar = ViewAxis(1:177, Axis(T_in = ViewAxis(1:177, Axis(layer_1 = ViewAxis(1:24, Axis(weight = ViewAxis(1:16, ShapedAxis((8, 2), NamedTuple())), bias = ViewAxis(17:24, ShapedAxis((8, 1), NamedTuple())))), layer_2 = ViewAxis(25:96, Axis(weight = ViewAxis(1:64, ShapedAxis((8, 8), NamedTuple())), bias = ViewAxis(65:72, ShapedAxis((8, 1), NamedTuple())))), layer_3 = ViewAxis(97:168, Axis(weight = ViewAxis(1:64, ShapedAxis((8, 8), NamedTuple())), bias = ViewAxis(65:72, ShapedAxis((8, 1), NamedTuple())))), layer_4 = ViewAxis(169:177, Axis(weight = ViewAxis(1:8, ShapedAxis((1, 8), NamedTuple())), bias = ViewAxis(9:9, ShapedAxis((1, 1), NamedTuple())))))),)), p = 178:180)}}})
    @ NLSolversBase C:\Users\loren\.julia\packages\NLSolversBase\kavn7\src\interface.jl:82
 [66] initial_state(method::BFGS{LineSearches.InitialStatic{Float64}, LineSearches.HagerZhang{Float64, Base.RefValue{Bool}}, Nothing, Nothing, Flat}, options::Optim.Options{Float64, OptimizationOptimJL.var"#_cb#14"{OptimizationOptimJL.OptimJLOptimizationCache{OptimizationFunction{false, Optimization.AutoZygote, OptimizationFunction{true, Optimization.AutoZygote, NeuralPDE.var"#full_loss_function#308"{NeuralPDE.var"#null_nonadaptive_loss#115", Vector{NeuralPDE.var"#74#75"{NeuralPDE.var"#209#210"{RuntimeGeneratedFunctions.RuntimeGeneratedFunction{(:cord, Symbol("##θ#293"), :phi, :derivative, :integral, :u, :p), NeuralPDE.var"#_RGF_ModTag", NeuralPDE.var"#_RGF_ModTag", (0x295b6845, 0xce1110b6, 0xa17beb73, 0xc449b733, 0xb3d6896e), Expr}, NeuralPDE.var"#12#13", NeuralPDE.var"#269#276"{NeuralPDE.var"#269#270#277"{typeof(NeuralPDE.numeric_derivative)}, Dict{Symbol, Int64}, Dict{Symbol, Int64}, GridTraining{Float64}}, typeof(NeuralPDE.numeric_derivative), Vector{NeuralPDE.Phi{Lux.Chain{NamedTuple{(:layer_1, :layer_2, :layer_3, :layer_4), Tuple{Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(identity), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}}}}, NamedTuple{(:layer_1, :layer_2, :layer_3, :layer_4), NTuple{4, NamedTuple{(), Tuple{}}}}}}, Vector{Float64}}, Matrix{Float64}}}, Vector{NeuralPDE.var"#74#75"{NeuralPDE.var"#209#210"{RuntimeGeneratedFunctions.RuntimeGeneratedFunction{(:cord, Symbol("##θ#293"), :phi, :derivative, :integral, :u, :p), NeuralPDE.var"#_RGF_ModTag", NeuralPDE.var"#_RGF_ModTag", (0xb5fb80bb, 0x61a62589, 0x2ec55669, 0x07bea97e, 0xb7ae3a52), Expr}, NeuralPDE.var"#12#13", NeuralPDE.var"#269#276"{NeuralPDE.var"#269#270#277"{typeof(NeuralPDE.numeric_derivative)}, Dict{Symbol, Int64}, Dict{Symbol, Int64}, GridTraining{Float64}}, typeof(NeuralPDE.numeric_derivative), Vector{NeuralPDE.Phi{Lux.Chain{NamedTuple{(:layer_1, :layer_2, :layer_3, :layer_4), Tuple{Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(identity), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}}}}, NamedTuple{(:layer_1, :layer_2, :layer_3, :layer_4), NTuple{4, NamedTuple{(), Tuple{}}}}}}, Vector{Float64}}, Matrix{Float64}}}, NeuralPDE.PINNRepresentation, Bool, Vector{Int64}, Int64, Vector{NeuralPDE.Phi{Lux.Chain{NamedTuple{(:layer_1, :layer_2, :layer_3, :layer_4), Tuple{Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(identity), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}}}}, NamedTuple{(:layer_1, :layer_2, :layer_3, :layer_4), NTuple{4, NamedTuple{(), Tuple{}}}}}}, Nothing, Bool, Vector{Float64}}, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, typeof(SciMLBase.DEFAULT_OBSERVED_NO_TIME), Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing}, Optimization.var"#228#237"{Optimization.var"#227#236"{OptimizationFunction{true, Optimization.AutoZygote, NeuralPDE.var"#full_loss_function#308"{NeuralPDE.var"#null_nonadaptive_loss#115", Vector{NeuralPDE.var"#74#75"{NeuralPDE.var"#209#210"{RuntimeGeneratedFunctions.RuntimeGeneratedFunction{(:cord, Symbol("##θ#293"), :phi, :derivative, :integral, :u, :p), NeuralPDE.var"#_RGF_ModTag", NeuralPDE.var"#_RGF_ModTag", (0x295b6845, 0xce1110b6, 0xa17beb73, 0xc449b733, 0xb3d6896e), Expr}, NeuralPDE.var"#12#13", NeuralPDE.var"#269#276"{NeuralPDE.var"#269#270#277"{typeof(NeuralPDE.numeric_derivative)}, Dict{Symbol, Int64}, Dict{Symbol, Int64}, GridTraining{Float64}}, typeof(NeuralPDE.numeric_derivative), Vector{NeuralPDE.Phi{Lux.Chain{NamedTuple{(:layer_1, :layer_2, :layer_3, :layer_4), Tuple{Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(identity), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}}}}, NamedTuple{(:layer_1, :layer_2, :layer_3, :layer_4), NTuple{4, NamedTuple{(), Tuple{}}}}}}, Vector{Float64}}, Matrix{Float64}}}, Vector{NeuralPDE.var"#74#75"{NeuralPDE.var"#209#210"{RuntimeGeneratedFunctions.RuntimeGeneratedFunction{(:cord, Symbol("##θ#293"), :phi, :derivative, :integral, :u, :p), NeuralPDE.var"#_RGF_ModTag", NeuralPDE.var"#_RGF_ModTag", (0xb5fb80bb, 0x61a62589, 0x2ec55669, 0x07bea97e, 0xb7ae3a52), Expr}, NeuralPDE.var"#12#13", NeuralPDE.var"#269#276"{NeuralPDE.var"#269#270#277"{typeof(NeuralPDE.numeric_derivative)}, Dict{Symbol, Int64}, Dict{Symbol, Int64}, GridTraining{Float64}}, typeof(NeuralPDE.numeric_derivative), Vector{NeuralPDE.Phi{Lux.Chain{NamedTuple{(:layer_1, :layer_2, :layer_3, :layer_4), Tuple{Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(identity), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}}}}, NamedTuple{(:layer_1, :layer_2, :layer_3, :layer_4), NTuple{4, NamedTuple{(), Tuple{}}}}}}, Vector{Float64}}, Matrix{Float64}}}, NeuralPDE.PINNRepresentation, Bool, Vector{Int64}, Int64, Vector{NeuralPDE.Phi{Lux.Chain{NamedTuple{(:layer_1, :layer_2, :layer_3, :layer_4), Tuple{Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(identity), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}}}}, NamedTuple{(:layer_1, :layer_2, :layer_3, :layer_4), NTuple{4, NamedTuple{(), Tuple{}}}}}}, Nothing, Bool, Vector{Float64}}, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, typeof(SciMLBase.DEFAULT_OBSERVED_NO_TIME), Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing}, Optimization.ReInitCache{ComponentArrays.ComponentVector{Float64, Vector{Float64}, Tuple{ComponentArrays.Axis{(depvar = ViewAxis(1:177, Axis(T_in = ViewAxis(1:177, Axis(layer_1 = ViewAxis(1:24, Axis(weight = ViewAxis(1:16, ShapedAxis((8, 2), NamedTuple())), bias = ViewAxis(17:24, ShapedAxis((8, 1), NamedTuple())))), layer_2 = ViewAxis(25:96, Axis(weight = ViewAxis(1:64, ShapedAxis((8, 8), NamedTuple())), bias = ViewAxis(65:72, ShapedAxis((8, 1), NamedTuple())))), layer_3 = ViewAxis(97:168, Axis(weight = ViewAxis(1:64, ShapedAxis((8, 8), NamedTuple())), bias = ViewAxis(65:72, ShapedAxis((8, 1), NamedTuple())))), layer_4 = ViewAxis(169:177, Axis(weight = ViewAxis(1:8, ShapedAxis((1, 8), NamedTuple())), bias = ViewAxis(9:9, ShapedAxis((1, 1), NamedTuple())))))),)), p = 178:180)}}}, SciMLBase.NullParameters}}}, Optimization.var"#231#240"{Optimization.var"#227#236"{OptimizationFunction{true, Optimization.AutoZygote, NeuralPDE.var"#full_loss_function#308"{NeuralPDE.var"#null_nonadaptive_loss#115", Vector{NeuralPDE.var"#74#75"{NeuralPDE.var"#209#210"{RuntimeGeneratedFunctions.RuntimeGeneratedFunction{(:cord, Symbol("##θ#293"), :phi, :derivative, :integral, :u, :p), NeuralPDE.var"#_RGF_ModTag", NeuralPDE.var"#_RGF_ModTag", (0x295b6845, 0xce1110b6, 0xa17beb73, 0xc449b733, 0xb3d6896e), Expr}, NeuralPDE.var"#12#13", NeuralPDE.var"#269#276"{NeuralPDE.var"#269#270#277"{typeof(NeuralPDE.numeric_derivative)}, Dict{Symbol, Int64}, Dict{Symbol, Int64}, GridTraining{Float64}}, typeof(NeuralPDE.numeric_derivative), Vector{NeuralPDE.Phi{Lux.Chain{NamedTuple{(:layer_1, :layer_2, :layer_3, :layer_4), Tuple{Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(identity), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}}}}, NamedTuple{(:layer_1, :layer_2, :layer_3, :layer_4), NTuple{4, NamedTuple{(), Tuple{}}}}}}, Vector{Float64}}, Matrix{Float64}}}, Vector{NeuralPDE.var"#74#75"{NeuralPDE.var"#209#210"{RuntimeGeneratedFunctions.RuntimeGeneratedFunction{(:cord, Symbol("##θ#293"), :phi, :derivative, :integral, :u, :p), NeuralPDE.var"#_RGF_ModTag", NeuralPDE.var"#_RGF_ModTag", (0xb5fb80bb, 0x61a62589, 0x2ec55669, 0x07bea97e, 0xb7ae3a52), Expr}, NeuralPDE.var"#12#13", NeuralPDE.var"#269#276"{NeuralPDE.var"#269#270#277"{typeof(NeuralPDE.numeric_derivative)}, Dict{Symbol, Int64}, Dict{Symbol, Int64}, GridTraining{Float64}}, typeof(NeuralPDE.numeric_derivative), Vector{NeuralPDE.Phi{Lux.Chain{NamedTuple{(:layer_1, :layer_2, :layer_3, :layer_4), Tuple{Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(identity), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}}}}, NamedTuple{(:layer_1, :layer_2, :layer_3, :layer_4), NTuple{4, NamedTuple{(), Tuple{}}}}}}, Vector{Float64}}, Matrix{Float64}}}, NeuralPDE.PINNRepresentation, Bool, Vector{Int64}, Int64, Vector{NeuralPDE.Phi{Lux.Chain{NamedTuple{(:layer_1, :layer_2, :layer_3, :layer_4), Tuple{Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(identity), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}}}}, NamedTuple{(:layer_1, :layer_2, :layer_3, :layer_4), NTuple{4, NamedTuple{(), Tuple{}}}}}}, Nothing, Bool, Vector{Float64}}, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, typeof(SciMLBase.DEFAULT_OBSERVED_NO_TIME), Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing}, Optimization.ReInitCache{ComponentArrays.ComponentVector{Float64, Vector{Float64}, Tuple{ComponentArrays.Axis{(depvar = ViewAxis(1:177, Axis(T_in = ViewAxis(1:177, Axis(layer_1 = ViewAxis(1:24, Axis(weight = ViewAxis(1:16, ShapedAxis((8, 2), NamedTuple())), bias = ViewAxis(17:24, ShapedAxis((8, 1), NamedTuple())))), layer_2 = ViewAxis(25:96, Axis(weight = ViewAxis(1:64, ShapedAxis((8, 8), NamedTuple())), bias = ViewAxis(65:72, ShapedAxis((8, 1), NamedTuple())))), layer_3 = ViewAxis(97:168, Axis(weight = ViewAxis(1:64, ShapedAxis((8, 8), NamedTuple())), bias = ViewAxis(65:72, ShapedAxis((8, 1), NamedTuple())))), layer_4 = ViewAxis(169:177, Axis(weight = ViewAxis(1:8, ShapedAxis((1, 8), NamedTuple())), bias = ViewAxis(9:9, ShapedAxis((1, 1), NamedTuple())))))),)), p = 178:180)}}}, SciMLBase.NullParameters}}}, Optimization.var"#235#244", Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, typeof(SciMLBase.DEFAULT_OBSERVED_NO_TIME), Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing}, Optimization.ReInitCache{ComponentArrays.ComponentVector{Float64, Vector{Float64}, Tuple{ComponentArrays.Axis{(depvar = ViewAxis(1:177, Axis(T_in = ViewAxis(1:177, Axis(layer_1 = ViewAxis(1:24, Axis(weight = ViewAxis(1:16, ShapedAxis((8, 2), NamedTuple())), bias = ViewAxis(17:24, ShapedAxis((8, 1), NamedTuple())))), layer_2 = ViewAxis(25:96, Axis(weight = ViewAxis(1:64, ShapedAxis((8, 8), NamedTuple())), bias = ViewAxis(65:72, ShapedAxis((8, 1), NamedTuple())))), layer_3 = ViewAxis(97:168, Axis(weight = ViewAxis(1:64, ShapedAxis((8, 8), NamedTuple())), bias = ViewAxis(65:72, ShapedAxis((8, 1), NamedTuple())))), layer_4 = ViewAxis(169:177, Axis(weight = ViewAxis(1:8, ShapedAxis((1, 8), NamedTuple())), bias = ViewAxis(9:9, ShapedAxis((1, 1), NamedTuple())))))),)), p = 178:180)}}}, SciMLBase.NullParameters}, Nothing, Nothing, Nothing, Nothing, Nothing, BFGS{LineSearches.InitialStatic{Float64}, LineSearches.HagerZhang{Float64, Base.RefValue{Bool}}, Nothing, Nothing, Flat}, Base.Iterators.Cycle{Tuple{Optimization.NullData}}, Bool, var"#9#10"}}}, d::TwiceDifferentiable{Float64, ComponentArrays.ComponentVector{Float64, Vector{Float64}, Tuple{ComponentArrays.Axis{(depvar = ViewAxis(1:177, Axis(T_in = ViewAxis(1:177, Axis(layer_1 = ViewAxis(1:24, Axis(weight = ViewAxis(1:16, ShapedAxis((8, 2), NamedTuple())), bias = ViewAxis(17:24, ShapedAxis((8, 1), NamedTuple())))), layer_2 = ViewAxis(25:96, Axis(weight = ViewAxis(1:64, ShapedAxis((8, 8), NamedTuple())), bias = ViewAxis(65:72, ShapedAxis((8, 1), NamedTuple())))), layer_3 = ViewAxis(97:168, Axis(weight = ViewAxis(1:64, ShapedAxis((8, 8), NamedTuple())), bias = ViewAxis(65:72, ShapedAxis((8, 1), NamedTuple())))), layer_4 = ViewAxis(169:177, Axis(weight = ViewAxis(1:8, ShapedAxis((1, 8), NamedTuple())), bias = ViewAxis(9:9, ShapedAxis((1, 1), NamedTuple())))))),)), p = 178:180)}}}, ComponentArrays.ComponentMatrix{Float64, Matrix{Float64}, Tuple{ComponentArrays.Axis{(depvar = ViewAxis(1:177, Axis(T_in = ViewAxis(1:177, Axis(layer_1 = ViewAxis(1:24, Axis(weight = ViewAxis(1:16, ShapedAxis((8, 2), NamedTuple())), bias = ViewAxis(17:24, ShapedAxis((8, 1), NamedTuple())))), layer_2 = ViewAxis(25:96, Axis(weight = ViewAxis(1:64, ShapedAxis((8, 8), NamedTuple())), bias = ViewAxis(65:72, ShapedAxis((8, 1), NamedTuple())))), layer_3 = ViewAxis(97:168, Axis(weight = ViewAxis(1:64, ShapedAxis((8, 8), NamedTuple())), bias = ViewAxis(65:72, ShapedAxis((8, 1), NamedTuple())))), layer_4 = ViewAxis(169:177, Axis(weight = ViewAxis(1:8, ShapedAxis((1, 8), NamedTuple())), bias = ViewAxis(9:9, ShapedAxis((1, 1), NamedTuple())))))),)), p = 178:180)}, ComponentArrays.Axis{(depvar = ViewAxis(1:177, Axis(T_in = ViewAxis(1:177, Axis(layer_1 = ViewAxis(1:24, Axis(weight = ViewAxis(1:16, ShapedAxis((8, 2), NamedTuple())), bias = ViewAxis(17:24, ShapedAxis((8, 1), NamedTuple())))), layer_2 = ViewAxis(25:96, Axis(weight = ViewAxis(1:64, ShapedAxis((8, 8), NamedTuple())), bias = ViewAxis(65:72, ShapedAxis((8, 1), NamedTuple())))), layer_3 = ViewAxis(97:168, Axis(weight = ViewAxis(1:64, ShapedAxis((8, 8), NamedTuple())), bias = ViewAxis(65:72, ShapedAxis((8, 1), NamedTuple())))), layer_4 = ViewAxis(169:177, Axis(weight = ViewAxis(1:8, ShapedAxis((1, 8), NamedTuple())), bias = ViewAxis(9:9, ShapedAxis((1, 1), NamedTuple())))))),)), p = 178:180)}}}, ComponentArrays.ComponentVector{Float64, Vector{Float64}, Tuple{ComponentArrays.Axis{(depvar = ViewAxis(1:177, Axis(T_in = ViewAxis(1:177, Axis(layer_1 = ViewAxis(1:24, Axis(weight = ViewAxis(1:16, ShapedAxis((8, 2), NamedTuple())), bias = ViewAxis(17:24, ShapedAxis((8, 1), NamedTuple())))), layer_2 = ViewAxis(25:96, Axis(weight = ViewAxis(1:64, ShapedAxis((8, 8), NamedTuple())), bias = ViewAxis(65:72, ShapedAxis((8, 1), NamedTuple())))), layer_3 = ViewAxis(97:168, Axis(weight = ViewAxis(1:64, ShapedAxis((8, 8), NamedTuple())), bias = ViewAxis(65:72, ShapedAxis((8, 1), NamedTuple())))), layer_4 = ViewAxis(169:177, Axis(weight = ViewAxis(1:8, ShapedAxis((1, 8), NamedTuple())), bias = ViewAxis(9:9, ShapedAxis((1, 1), NamedTuple())))))),)), p = 178:180)}}}}, initial_x::ComponentArrays.ComponentVector{Float64, Vector{Float64}, Tuple{ComponentArrays.Axis{(depvar = ViewAxis(1:177, Axis(T_in = ViewAxis(1:177, Axis(layer_1 = ViewAxis(1:24, Axis(weight = ViewAxis(1:16, ShapedAxis((8, 2), NamedTuple())), bias = ViewAxis(17:24, ShapedAxis((8, 1), NamedTuple())))), layer_2 = ViewAxis(25:96, Axis(weight = ViewAxis(1:64, ShapedAxis((8, 8), NamedTuple())), bias = ViewAxis(65:72, ShapedAxis((8, 1), NamedTuple())))), layer_3 = ViewAxis(97:168, Axis(weight = ViewAxis(1:64, ShapedAxis((8, 8), NamedTuple())), bias = ViewAxis(65:72, ShapedAxis((8, 1), NamedTuple())))), layer_4 = ViewAxis(169:177, Axis(weight = ViewAxis(1:8, ShapedAxis((1, 8), NamedTuple())), bias = ViewAxis(9:9, ShapedAxis((1, 1), NamedTuple())))))),)), p = 178:180)}}})
    @ Optim C:\Users\loren\.julia\packages\Optim\V8ZEC\src\multivariate\solvers\first_order\bfgs.jl:94
 [67] optimize(d::TwiceDifferentiable{Float64, ComponentArrays.ComponentVector{Float64, Vector{Float64}, Tuple{ComponentArrays.Axis{(depvar = ViewAxis(1:177, Axis(T_in = ViewAxis(1:177, Axis(layer_1 = ViewAxis(1:24, Axis(weight = ViewAxis(1:16, ShapedAxis((8, 2), NamedTuple())), bias = ViewAxis(17:24, ShapedAxis((8, 1), NamedTuple())))), layer_2 = ViewAxis(25:96, Axis(weight = ViewAxis(1:64, ShapedAxis((8, 8), NamedTuple())), bias = ViewAxis(65:72, ShapedAxis((8, 1), NamedTuple())))), layer_3 = ViewAxis(97:168, Axis(weight = ViewAxis(1:64, ShapedAxis((8, 8), NamedTuple())), bias = ViewAxis(65:72, ShapedAxis((8, 1), NamedTuple())))), layer_4 = ViewAxis(169:177, Axis(weight = ViewAxis(1:8, ShapedAxis((1, 8), NamedTuple())), bias = ViewAxis(9:9, ShapedAxis((1, 1), NamedTuple())))))),)), p = 178:180)}}}, ComponentArrays.ComponentMatrix{Float64, Matrix{Float64}, Tuple{ComponentArrays.Axis{(depvar = ViewAxis(1:177, Axis(T_in = ViewAxis(1:177, Axis(layer_1 = ViewAxis(1:24, Axis(weight = ViewAxis(1:16, ShapedAxis((8, 2), NamedTuple())), bias = ViewAxis(17:24, ShapedAxis((8, 1), NamedTuple())))), layer_2 = ViewAxis(25:96, Axis(weight = ViewAxis(1:64, ShapedAxis((8, 8), NamedTuple())), bias = ViewAxis(65:72, ShapedAxis((8, 1), NamedTuple())))), layer_3 = ViewAxis(97:168, Axis(weight = ViewAxis(1:64, ShapedAxis((8, 8), NamedTuple())), bias = ViewAxis(65:72, ShapedAxis((8, 1), NamedTuple())))), layer_4 = ViewAxis(169:177, Axis(weight = ViewAxis(1:8, ShapedAxis((1, 8), NamedTuple())), bias = ViewAxis(9:9, ShapedAxis((1, 1), NamedTuple())))))),)), p = 178:180)}, ComponentArrays.Axis{(depvar = ViewAxis(1:177, Axis(T_in = ViewAxis(1:177, Axis(layer_1 = ViewAxis(1:24, Axis(weight = ViewAxis(1:16, ShapedAxis((8, 2), NamedTuple())), bias = ViewAxis(17:24, ShapedAxis((8, 1), NamedTuple())))), layer_2 = ViewAxis(25:96, Axis(weight = ViewAxis(1:64, ShapedAxis((8, 8), NamedTuple())), bias = ViewAxis(65:72, ShapedAxis((8, 1), NamedTuple())))), layer_3 = ViewAxis(97:168, Axis(weight = ViewAxis(1:64, ShapedAxis((8, 8), NamedTuple())), bias = ViewAxis(65:72, ShapedAxis((8, 1), NamedTuple())))), layer_4 = ViewAxis(169:177, Axis(weight = ViewAxis(1:8, ShapedAxis((1, 8), NamedTuple())), bias = ViewAxis(9:9, ShapedAxis((1, 1), NamedTuple())))))),)), p = 178:180)}}}, ComponentArrays.ComponentVector{Float64, Vector{Float64}, Tuple{ComponentArrays.Axis{(depvar = ViewAxis(1:177, Axis(T_in = ViewAxis(1:177, Axis(layer_1 = ViewAxis(1:24, Axis(weight = ViewAxis(1:16, ShapedAxis((8, 2), NamedTuple())), bias = ViewAxis(17:24, ShapedAxis((8, 1), NamedTuple())))), layer_2 = ViewAxis(25:96, Axis(weight = ViewAxis(1:64, ShapedAxis((8, 8), NamedTuple())), bias = ViewAxis(65:72, ShapedAxis((8, 1), NamedTuple())))), layer_3 = ViewAxis(97:168, Axis(weight = ViewAxis(1:64, ShapedAxis((8, 8), NamedTuple())), bias = ViewAxis(65:72, ShapedAxis((8, 1), NamedTuple())))), layer_4 = ViewAxis(169:177, Axis(weight = ViewAxis(1:8, ShapedAxis((1, 8), NamedTuple())), bias = ViewAxis(9:9, ShapedAxis((1, 1), NamedTuple())))))),)), p = 178:180)}}}}, initial_x::ComponentArrays.ComponentVector{Float64, Vector{Float64}, Tuple{ComponentArrays.Axis{(depvar = ViewAxis(1:177, Axis(T_in = ViewAxis(1:177, Axis(layer_1 = ViewAxis(1:24, Axis(weight = ViewAxis(1:16, ShapedAxis((8, 2), NamedTuple())), bias = ViewAxis(17:24, ShapedAxis((8, 1), NamedTuple())))), layer_2 = ViewAxis(25:96, Axis(weight = ViewAxis(1:64, ShapedAxis((8, 8), NamedTuple())), bias = ViewAxis(65:72, ShapedAxis((8, 1), NamedTuple())))), layer_3 = ViewAxis(97:168, Axis(weight = ViewAxis(1:64, ShapedAxis((8, 8), NamedTuple())), bias = ViewAxis(65:72, ShapedAxis((8, 1), NamedTuple())))), layer_4 = ViewAxis(169:177, Axis(weight = ViewAxis(1:8, ShapedAxis((1, 8), NamedTuple())), bias = ViewAxis(9:9, ShapedAxis((1, 1), NamedTuple())))))),)), p = 178:180)}}}, method::BFGS{LineSearches.InitialStatic{Float64}, LineSearches.HagerZhang{Float64, Base.RefValue{Bool}}, Nothing, Nothing, Flat}, options::Optim.Options{Float64, OptimizationOptimJL.var"#_cb#14"{OptimizationOptimJL.OptimJLOptimizationCache{OptimizationFunction{false, Optimization.AutoZygote, OptimizationFunction{true, Optimization.AutoZygote, NeuralPDE.var"#full_loss_function#308"{NeuralPDE.var"#null_nonadaptive_loss#115", Vector{NeuralPDE.var"#74#75"{NeuralPDE.var"#209#210"{RuntimeGeneratedFunctions.RuntimeGeneratedFunction{(:cord, Symbol("##θ#293"), :phi, :derivative, :integral, :u, :p), NeuralPDE.var"#_RGF_ModTag", NeuralPDE.var"#_RGF_ModTag", (0x295b6845, 0xce1110b6, 0xa17beb73, 0xc449b733, 0xb3d6896e), Expr}, NeuralPDE.var"#12#13", NeuralPDE.var"#269#276"{NeuralPDE.var"#269#270#277"{typeof(NeuralPDE.numeric_derivative)}, Dict{Symbol, Int64}, Dict{Symbol, Int64}, GridTraining{Float64}}, typeof(NeuralPDE.numeric_derivative), Vector{NeuralPDE.Phi{Lux.Chain{NamedTuple{(:layer_1, :layer_2, :layer_3, :layer_4), Tuple{Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(identity), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}}}}, NamedTuple{(:layer_1, :layer_2, :layer_3, :layer_4), NTuple{4, NamedTuple{(), Tuple{}}}}}}, Vector{Float64}}, Matrix{Float64}}}, Vector{NeuralPDE.var"#74#75"{NeuralPDE.var"#209#210"{RuntimeGeneratedFunctions.RuntimeGeneratedFunction{(:cord, Symbol("##θ#293"), :phi, :derivative, :integral, :u, :p), NeuralPDE.var"#_RGF_ModTag", NeuralPDE.var"#_RGF_ModTag", (0xb5fb80bb, 0x61a62589, 0x2ec55669, 0x07bea97e, 0xb7ae3a52), Expr}, NeuralPDE.var"#12#13", NeuralPDE.var"#269#276"{NeuralPDE.var"#269#270#277"{typeof(NeuralPDE.numeric_derivative)}, Dict{Symbol, Int64}, Dict{Symbol, Int64}, GridTraining{Float64}}, typeof(NeuralPDE.numeric_derivative), Vector{NeuralPDE.Phi{Lux.Chain{NamedTuple{(:layer_1, :layer_2, :layer_3, :layer_4), Tuple{Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(identity), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}}}}, NamedTuple{(:layer_1, :layer_2, :layer_3, :layer_4), NTuple{4, NamedTuple{(), Tuple{}}}}}}, Vector{Float64}}, Matrix{Float64}}}, NeuralPDE.PINNRepresentation, Bool, Vector{Int64}, Int64, Vector{NeuralPDE.Phi{Lux.Chain{NamedTuple{(:layer_1, :layer_2, :layer_3, :layer_4), Tuple{Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(identity), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}}}}, NamedTuple{(:layer_1, :layer_2, :layer_3, :layer_4), NTuple{4, NamedTuple{(), Tuple{}}}}}}, Nothing, Bool, Vector{Float64}}, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, typeof(SciMLBase.DEFAULT_OBSERVED_NO_TIME), Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing}, Optimization.var"#228#237"{Optimization.var"#227#236"{OptimizationFunction{true, Optimization.AutoZygote, NeuralPDE.var"#full_loss_function#308"{NeuralPDE.var"#null_nonadaptive_loss#115", Vector{NeuralPDE.var"#74#75"{NeuralPDE.var"#209#210"{RuntimeGeneratedFunctions.RuntimeGeneratedFunction{(:cord, Symbol("##θ#293"), :phi, :derivative, :integral, :u, :p), NeuralPDE.var"#_RGF_ModTag", NeuralPDE.var"#_RGF_ModTag", (0x295b6845, 0xce1110b6, 0xa17beb73, 0xc449b733, 0xb3d6896e), Expr}, NeuralPDE.var"#12#13", NeuralPDE.var"#269#276"{NeuralPDE.var"#269#270#277"{typeof(NeuralPDE.numeric_derivative)}, Dict{Symbol, Int64}, Dict{Symbol, Int64}, GridTraining{Float64}}, typeof(NeuralPDE.numeric_derivative), Vector{NeuralPDE.Phi{Lux.Chain{NamedTuple{(:layer_1, :layer_2, :layer_3, :layer_4), Tuple{Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(identity), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}}}}, NamedTuple{(:layer_1, :layer_2, :layer_3, :layer_4), NTuple{4, NamedTuple{(), Tuple{}}}}}}, Vector{Float64}}, Matrix{Float64}}}, Vector{NeuralPDE.var"#74#75"{NeuralPDE.var"#209#210"{RuntimeGeneratedFunctions.RuntimeGeneratedFunction{(:cord, Symbol("##θ#293"), :phi, :derivative, :integral, :u, :p), NeuralPDE.var"#_RGF_ModTag", NeuralPDE.var"#_RGF_ModTag", (0xb5fb80bb, 0x61a62589, 0x2ec55669, 0x07bea97e, 0xb7ae3a52), Expr}, NeuralPDE.var"#12#13", NeuralPDE.var"#269#276"{NeuralPDE.var"#269#270#277"{typeof(NeuralPDE.numeric_derivative)}, Dict{Symbol, Int64}, Dict{Symbol, Int64}, GridTraining{Float64}}, typeof(NeuralPDE.numeric_derivative), Vector{NeuralPDE.Phi{Lux.Chain{NamedTuple{(:layer_1, :layer_2, :layer_3, :layer_4), Tuple{Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(identity), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}}}}, NamedTuple{(:layer_1, :layer_2, :layer_3, :layer_4), NTuple{4, NamedTuple{(), Tuple{}}}}}}, Vector{Float64}}, Matrix{Float64}}}, NeuralPDE.PINNRepresentation, Bool, Vector{Int64}, Int64, Vector{NeuralPDE.Phi{Lux.Chain{NamedTuple{(:layer_1, :layer_2, :layer_3, :layer_4), Tuple{Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(identity), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}}}}, NamedTuple{(:layer_1, :layer_2, :layer_3, :layer_4), NTuple{4, NamedTuple{(), Tuple{}}}}}}, Nothing, Bool, Vector{Float64}}, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, typeof(SciMLBase.DEFAULT_OBSERVED_NO_TIME), Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing}, Optimization.ReInitCache{ComponentArrays.ComponentVector{Float64, Vector{Float64}, Tuple{ComponentArrays.Axis{(depvar = ViewAxis(1:177, Axis(T_in = ViewAxis(1:177, Axis(layer_1 = ViewAxis(1:24, Axis(weight = ViewAxis(1:16, ShapedAxis((8, 2), NamedTuple())), bias = ViewAxis(17:24, ShapedAxis((8, 1), NamedTuple())))), layer_2 = ViewAxis(25:96, Axis(weight = ViewAxis(1:64, ShapedAxis((8, 8), NamedTuple())), bias = ViewAxis(65:72, ShapedAxis((8, 1), NamedTuple())))), layer_3 = ViewAxis(97:168, Axis(weight = ViewAxis(1:64, ShapedAxis((8, 8), NamedTuple())), bias = ViewAxis(65:72, ShapedAxis((8, 1), NamedTuple())))), layer_4 = ViewAxis(169:177, Axis(weight = ViewAxis(1:8, ShapedAxis((1, 8), NamedTuple())), bias = ViewAxis(9:9, ShapedAxis((1, 1), NamedTuple())))))),)), p = 178:180)}}}, SciMLBase.NullParameters}}}, Optimization.var"#231#240"{Optimization.var"#227#236"{OptimizationFunction{true, Optimization.AutoZygote, NeuralPDE.var"#full_loss_function#308"{NeuralPDE.var"#null_nonadaptive_loss#115", Vector{NeuralPDE.var"#74#75"{NeuralPDE.var"#209#210"{RuntimeGeneratedFunctions.RuntimeGeneratedFunction{(:cord, Symbol("##θ#293"), :phi, :derivative, :integral, :u, :p), NeuralPDE.var"#_RGF_ModTag", NeuralPDE.var"#_RGF_ModTag", (0x295b6845, 0xce1110b6, 0xa17beb73, 0xc449b733, 0xb3d6896e), Expr}, NeuralPDE.var"#12#13", NeuralPDE.var"#269#276"{NeuralPDE.var"#269#270#277"{typeof(NeuralPDE.numeric_derivative)}, Dict{Symbol, Int64}, Dict{Symbol, Int64}, GridTraining{Float64}}, typeof(NeuralPDE.numeric_derivative), Vector{NeuralPDE.Phi{Lux.Chain{NamedTuple{(:layer_1, :layer_2, :layer_3, :layer_4), Tuple{Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(identity), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}}}}, NamedTuple{(:layer_1, :layer_2, :layer_3, :layer_4), NTuple{4, NamedTuple{(), Tuple{}}}}}}, Vector{Float64}}, Matrix{Float64}}}, Vector{NeuralPDE.var"#74#75"{NeuralPDE.var"#209#210"{RuntimeGeneratedFunctions.RuntimeGeneratedFunction{(:cord, Symbol("##θ#293"), :phi, :derivative, :integral, :u, :p), NeuralPDE.var"#_RGF_ModTag", NeuralPDE.var"#_RGF_ModTag", (0xb5fb80bb, 0x61a62589, 0x2ec55669, 0x07bea97e, 0xb7ae3a52), Expr}, NeuralPDE.var"#12#13", NeuralPDE.var"#269#276"{NeuralPDE.var"#269#270#277"{typeof(NeuralPDE.numeric_derivative)}, Dict{Symbol, Int64}, Dict{Symbol, Int64}, GridTraining{Float64}}, typeof(NeuralPDE.numeric_derivative), Vector{NeuralPDE.Phi{Lux.Chain{NamedTuple{(:layer_1, :layer_2, :layer_3, :layer_4), Tuple{Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(identity), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}}}}, NamedTuple{(:layer_1, :layer_2, :layer_3, :layer_4), NTuple{4, NamedTuple{(), Tuple{}}}}}}, Vector{Float64}}, Matrix{Float64}}}, NeuralPDE.PINNRepresentation, Bool, Vector{Int64}, Int64, Vector{NeuralPDE.Phi{Lux.Chain{NamedTuple{(:layer_1, :layer_2, :layer_3, :layer_4), Tuple{Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(identity), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}}}}, NamedTuple{(:layer_1, :layer_2, :layer_3, :layer_4), NTuple{4, NamedTuple{(), Tuple{}}}}}}, Nothing, Bool, Vector{Float64}}, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, typeof(SciMLBase.DEFAULT_OBSERVED_NO_TIME), Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing}, Optimization.ReInitCache{ComponentArrays.ComponentVector{Float64, Vector{Float64}, Tuple{ComponentArrays.Axis{(depvar = ViewAxis(1:177, Axis(T_in = ViewAxis(1:177, Axis(layer_1 = ViewAxis(1:24, Axis(weight = ViewAxis(1:16, ShapedAxis((8, 2), NamedTuple())), bias = ViewAxis(17:24, ShapedAxis((8, 1), NamedTuple())))), layer_2 = ViewAxis(25:96, Axis(weight = ViewAxis(1:64, ShapedAxis((8, 8), NamedTuple())), bias = ViewAxis(65:72, ShapedAxis((8, 1), NamedTuple())))), layer_3 = ViewAxis(97:168, Axis(weight = ViewAxis(1:64, ShapedAxis((8, 8), NamedTuple())), bias = ViewAxis(65:72, ShapedAxis((8, 1), NamedTuple())))), layer_4 = ViewAxis(169:177, Axis(weight = ViewAxis(1:8, ShapedAxis((1, 8), NamedTuple())), bias = ViewAxis(9:9, ShapedAxis((1, 1), NamedTuple())))))),)), p = 178:180)}}}, SciMLBase.NullParameters}}}, Optimization.var"#235#244", Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, typeof(SciMLBase.DEFAULT_OBSERVED_NO_TIME), Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing}, Optimization.ReInitCache{ComponentArrays.ComponentVector{Float64, Vector{Float64}, Tuple{ComponentArrays.Axis{(depvar = ViewAxis(1:177, Axis(T_in = ViewAxis(1:177, Axis(layer_1 = ViewAxis(1:24, Axis(weight = ViewAxis(1:16, ShapedAxis((8, 2), NamedTuple())), bias = ViewAxis(17:24, ShapedAxis((8, 1), NamedTuple())))), layer_2 = ViewAxis(25:96, Axis(weight = ViewAxis(1:64, ShapedAxis((8, 8), NamedTuple())), bias = ViewAxis(65:72, ShapedAxis((8, 1), NamedTuple())))), layer_3 = ViewAxis(97:168, Axis(weight = ViewAxis(1:64, ShapedAxis((8, 8), NamedTuple())), bias = ViewAxis(65:72, ShapedAxis((8, 1), NamedTuple())))), layer_4 = ViewAxis(169:177, Axis(weight = ViewAxis(1:8, ShapedAxis((1, 8), NamedTuple())), bias = ViewAxis(9:9, ShapedAxis((1, 1), NamedTuple())))))),)), p = 178:180)}}}, SciMLBase.NullParameters}, Nothing, Nothing, Nothing, Nothing, Nothing, BFGS{LineSearches.InitialStatic{Float64}, LineSearches.HagerZhang{Float64, Base.RefValue{Bool}}, Nothing, Nothing, Flat}, Base.Iterators.Cycle{Tuple{Optimization.NullData}}, Bool, var"#9#10"}}})
    @ Optim C:\Users\loren\.julia\packages\Optim\V8ZEC\src\multivariate\optimize\optimize.jl:36
 [68] __solve(cache::OptimizationOptimJL.OptimJLOptimizationCache{OptimizationFunction{false, Optimization.AutoZygote, OptimizationFunction{true, Optimization.AutoZygote, NeuralPDE.var"#full_loss_function#308"{NeuralPDE.var"#null_nonadaptive_loss#115", Vector{NeuralPDE.var"#74#75"{NeuralPDE.var"#209#210"{RuntimeGeneratedFunctions.RuntimeGeneratedFunction{(:cord, Symbol("##θ#293"), :phi, :derivative, :integral, :u, :p), NeuralPDE.var"#_RGF_ModTag", NeuralPDE.var"#_RGF_ModTag", (0x295b6845, 0xce1110b6, 0xa17beb73, 0xc449b733, 0xb3d6896e), Expr}, NeuralPDE.var"#12#13", NeuralPDE.var"#269#276"{NeuralPDE.var"#269#270#277"{typeof(NeuralPDE.numeric_derivative)}, Dict{Symbol, Int64}, Dict{Symbol, Int64}, GridTraining{Float64}}, typeof(NeuralPDE.numeric_derivative), Vector{NeuralPDE.Phi{Lux.Chain{NamedTuple{(:layer_1, :layer_2, :layer_3, :layer_4), Tuple{Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(identity), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}}}}, NamedTuple{(:layer_1, :layer_2, :layer_3, :layer_4), NTuple{4, NamedTuple{(), Tuple{}}}}}}, Vector{Float64}}, Matrix{Float64}}}, Vector{NeuralPDE.var"#74#75"{NeuralPDE.var"#209#210"{RuntimeGeneratedFunctions.RuntimeGeneratedFunction{(:cord, Symbol("##θ#293"), :phi, :derivative, :integral, :u, :p), NeuralPDE.var"#_RGF_ModTag", NeuralPDE.var"#_RGF_ModTag", (0xb5fb80bb, 0x61a62589, 0x2ec55669, 0x07bea97e, 0xb7ae3a52), Expr}, NeuralPDE.var"#12#13", NeuralPDE.var"#269#276"{NeuralPDE.var"#269#270#277"{typeof(NeuralPDE.numeric_derivative)}, Dict{Symbol, Int64}, Dict{Symbol, Int64}, GridTraining{Float64}}, typeof(NeuralPDE.numeric_derivative), Vector{NeuralPDE.Phi{Lux.Chain{NamedTuple{(:layer_1, :layer_2, :layer_3, :layer_4), Tuple{Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(identity), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}}}}, NamedTuple{(:layer_1, :layer_2, :layer_3, :layer_4), NTuple{4, NamedTuple{(), Tuple{}}}}}}, Vector{Float64}}, Matrix{Float64}}}, NeuralPDE.PINNRepresentation, Bool, Vector{Int64}, Int64, Vector{NeuralPDE.Phi{Lux.Chain{NamedTuple{(:layer_1, :layer_2, :layer_3, :layer_4), Tuple{Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(identity), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}}}}, NamedTuple{(:layer_1, :layer_2, :layer_3, :layer_4), NTuple{4, NamedTuple{(), Tuple{}}}}}}, Nothing, Bool, Vector{Float64}}, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, typeof(SciMLBase.DEFAULT_OBSERVED_NO_TIME), Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing}, Optimization.var"#228#237"{Optimization.var"#227#236"{OptimizationFunction{true, Optimization.AutoZygote, NeuralPDE.var"#full_loss_function#308"{NeuralPDE.var"#null_nonadaptive_loss#115", Vector{NeuralPDE.var"#74#75"{NeuralPDE.var"#209#210"{RuntimeGeneratedFunctions.RuntimeGeneratedFunction{(:cord, Symbol("##θ#293"), :phi, :derivative, :integral, :u, :p), NeuralPDE.var"#_RGF_ModTag", NeuralPDE.var"#_RGF_ModTag", (0x295b6845, 0xce1110b6, 0xa17beb73, 0xc449b733, 0xb3d6896e), Expr}, NeuralPDE.var"#12#13", NeuralPDE.var"#269#276"{NeuralPDE.var"#269#270#277"{typeof(NeuralPDE.numeric_derivative)}, Dict{Symbol, Int64}, Dict{Symbol, Int64}, GridTraining{Float64}}, typeof(NeuralPDE.numeric_derivative), Vector{NeuralPDE.Phi{Lux.Chain{NamedTuple{(:layer_1, :layer_2, :layer_3, :layer_4), Tuple{Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(identity), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}}}}, NamedTuple{(:layer_1, :layer_2, :layer_3, :layer_4), NTuple{4, NamedTuple{(), Tuple{}}}}}}, Vector{Float64}}, Matrix{Float64}}}, Vector{NeuralPDE.var"#74#75"{NeuralPDE.var"#209#210"{RuntimeGeneratedFunctions.RuntimeGeneratedFunction{(:cord, Symbol("##θ#293"), :phi, :derivative, :integral, :u, :p), NeuralPDE.var"#_RGF_ModTag", NeuralPDE.var"#_RGF_ModTag", (0xb5fb80bb, 0x61a62589, 0x2ec55669, 0x07bea97e, 0xb7ae3a52), Expr}, NeuralPDE.var"#12#13", NeuralPDE.var"#269#276"{NeuralPDE.var"#269#270#277"{typeof(NeuralPDE.numeric_derivative)}, Dict{Symbol, Int64}, Dict{Symbol, Int64}, GridTraining{Float64}}, typeof(NeuralPDE.numeric_derivative), Vector{NeuralPDE.Phi{Lux.Chain{NamedTuple{(:layer_1, :layer_2, :layer_3, :layer_4), Tuple{Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(identity), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}}}}, NamedTuple{(:layer_1, :layer_2, :layer_3, :layer_4), NTuple{4, NamedTuple{(), Tuple{}}}}}}, Vector{Float64}}, Matrix{Float64}}}, NeuralPDE.PINNRepresentation, Bool, Vector{Int64}, Int64, Vector{NeuralPDE.Phi{Lux.Chain{NamedTuple{(:layer_1, :layer_2, :layer_3, :layer_4), Tuple{Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(identity), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}}}}, NamedTuple{(:layer_1, :layer_2, :layer_3, :layer_4), NTuple{4, NamedTuple{(), Tuple{}}}}}}, Nothing, Bool, Vector{Float64}}, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, typeof(SciMLBase.DEFAULT_OBSERVED_NO_TIME), Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing}, Optimization.ReInitCache{ComponentArrays.ComponentVector{Float64, Vector{Float64}, Tuple{ComponentArrays.Axis{(depvar = ViewAxis(1:177, Axis(T_in = ViewAxis(1:177, Axis(layer_1 = ViewAxis(1:24, Axis(weight = ViewAxis(1:16, ShapedAxis((8, 2), NamedTuple())), bias = ViewAxis(17:24, ShapedAxis((8, 1), NamedTuple())))), layer_2 = ViewAxis(25:96, Axis(weight = ViewAxis(1:64, ShapedAxis((8, 8), NamedTuple())), bias = ViewAxis(65:72, ShapedAxis((8, 1), NamedTuple())))), layer_3 = ViewAxis(97:168, Axis(weight = ViewAxis(1:64, ShapedAxis((8, 8), NamedTuple())), bias = ViewAxis(65:72, ShapedAxis((8, 1), NamedTuple())))), layer_4 = ViewAxis(169:177, Axis(weight = ViewAxis(1:8, ShapedAxis((1, 8), NamedTuple())), bias = ViewAxis(9:9, ShapedAxis((1, 1), NamedTuple())))))),)), p = 178:180)}}}, SciMLBase.NullParameters}}}, Optimization.var"#231#240"{Optimization.var"#227#236"{OptimizationFunction{true, Optimization.AutoZygote, NeuralPDE.var"#full_loss_function#308"{NeuralPDE.var"#null_nonadaptive_loss#115", Vector{NeuralPDE.var"#74#75"{NeuralPDE.var"#209#210"{RuntimeGeneratedFunctions.RuntimeGeneratedFunction{(:cord, Symbol("##θ#293"), :phi, :derivative, :integral, :u, :p), NeuralPDE.var"#_RGF_ModTag", NeuralPDE.var"#_RGF_ModTag", (0x295b6845, 0xce1110b6, 0xa17beb73, 0xc449b733, 0xb3d6896e), Expr}, NeuralPDE.var"#12#13", NeuralPDE.var"#269#276"{NeuralPDE.var"#269#270#277"{typeof(NeuralPDE.numeric_derivative)}, Dict{Symbol, Int64}, Dict{Symbol, Int64}, GridTraining{Float64}}, typeof(NeuralPDE.numeric_derivative), Vector{NeuralPDE.Phi{Lux.Chain{NamedTuple{(:layer_1, :layer_2, :layer_3, :layer_4), Tuple{Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(identity), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}}}}, NamedTuple{(:layer_1, :layer_2, :layer_3, :layer_4), NTuple{4, NamedTuple{(), Tuple{}}}}}}, Vector{Float64}}, Matrix{Float64}}}, Vector{NeuralPDE.var"#74#75"{NeuralPDE.var"#209#210"{RuntimeGeneratedFunctions.RuntimeGeneratedFunction{(:cord, Symbol("##θ#293"), :phi, :derivative, :integral, :u, :p), NeuralPDE.var"#_RGF_ModTag", NeuralPDE.var"#_RGF_ModTag", (0xb5fb80bb, 0x61a62589, 0x2ec55669, 0x07bea97e, 0xb7ae3a52), Expr}, NeuralPDE.var"#12#13", NeuralPDE.var"#269#276"{NeuralPDE.var"#269#270#277"{typeof(NeuralPDE.numeric_derivative)}, Dict{Symbol, Int64}, Dict{Symbol, Int64}, GridTraining{Float64}}, typeof(NeuralPDE.numeric_derivative), Vector{NeuralPDE.Phi{Lux.Chain{NamedTuple{(:layer_1, :layer_2, :layer_3, :layer_4), Tuple{Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(identity), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}}}}, NamedTuple{(:layer_1, :layer_2, :layer_3, :layer_4), NTuple{4, NamedTuple{(), Tuple{}}}}}}, Vector{Float64}}, Matrix{Float64}}}, NeuralPDE.PINNRepresentation, Bool, Vector{Int64}, Int64, Vector{NeuralPDE.Phi{Lux.Chain{NamedTuple{(:layer_1, :layer_2, :layer_3, :layer_4), Tuple{Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(identity), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}}}}, NamedTuple{(:layer_1, :layer_2, :layer_3, :layer_4), NTuple{4, NamedTuple{(), Tuple{}}}}}}, Nothing, Bool, Vector{Float64}}, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, typeof(SciMLBase.DEFAULT_OBSERVED_NO_TIME), Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing}, Optimization.ReInitCache{ComponentArrays.ComponentVector{Float64, Vector{Float64}, Tuple{ComponentArrays.Axis{(depvar = ViewAxis(1:177, Axis(T_in = ViewAxis(1:177, Axis(layer_1 = ViewAxis(1:24, Axis(weight = ViewAxis(1:16, ShapedAxis((8, 2), NamedTuple())), bias = ViewAxis(17:24, ShapedAxis((8, 1), NamedTuple())))), layer_2 = ViewAxis(25:96, Axis(weight = ViewAxis(1:64, ShapedAxis((8, 8), NamedTuple())), bias = ViewAxis(65:72, ShapedAxis((8, 1), NamedTuple())))), layer_3 = ViewAxis(97:168, Axis(weight = ViewAxis(1:64, ShapedAxis((8, 8), NamedTuple())), bias = ViewAxis(65:72, ShapedAxis((8, 1), NamedTuple())))), layer_4 = ViewAxis(169:177, Axis(weight = ViewAxis(1:8, ShapedAxis((1, 8), NamedTuple())), bias = ViewAxis(9:9, ShapedAxis((1, 1), NamedTuple())))))),)), p = 178:180)}}}, SciMLBase.NullParameters}}}, Optimization.var"#235#244", Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, typeof(SciMLBase.DEFAULT_OBSERVED_NO_TIME), Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing}, Optimization.ReInitCache{ComponentArrays.ComponentVector{Float64, Vector{Float64}, Tuple{ComponentArrays.Axis{(depvar = ViewAxis(1:177, Axis(T_in = ViewAxis(1:177, Axis(layer_1 = ViewAxis(1:24, Axis(weight = ViewAxis(1:16, ShapedAxis((8, 2), NamedTuple())), bias = ViewAxis(17:24, ShapedAxis((8, 1), NamedTuple())))), layer_2 = ViewAxis(25:96, Axis(weight = ViewAxis(1:64, ShapedAxis((8, 8), NamedTuple())), bias = ViewAxis(65:72, ShapedAxis((8, 1), NamedTuple())))), layer_3 = ViewAxis(97:168, Axis(weight = ViewAxis(1:64, ShapedAxis((8, 8), NamedTuple())), bias = ViewAxis(65:72, ShapedAxis((8, 1), NamedTuple())))), layer_4 = ViewAxis(169:177, Axis(weight = ViewAxis(1:8, ShapedAxis((1, 8), NamedTuple())), bias = ViewAxis(9:9, ShapedAxis((1, 1), NamedTuple())))))),)), p = 178:180)}}}, SciMLBase.NullParameters}, Nothing, Nothing, Nothing, Nothing, Nothing, BFGS{LineSearches.InitialStatic{Float64}, LineSearches.HagerZhang{Float64, Base.RefValue{Bool}}, Nothing, Nothing, Flat}, Base.Iterators.Cycle{Tuple{Optimization.NullData}}, Bool, var"#9#10"})
    @ OptimizationOptimJL C:\Users\loren\.julia\packages\OptimizationOptimJL\uRfW9\src\OptimizationOptimJL.jl:249
 [69] solve!(cache::OptimizationOptimJL.OptimJLOptimizationCache{OptimizationFunction{false, Optimization.AutoZygote, OptimizationFunction{true, Optimization.AutoZygote, NeuralPDE.var"#full_loss_function#308"{NeuralPDE.var"#null_nonadaptive_loss#115", Vector{NeuralPDE.var"#74#75"{NeuralPDE.var"#209#210"{RuntimeGeneratedFunctions.RuntimeGeneratedFunction{(:cord, Symbol("##θ#293"), :phi, :derivative, :integral, :u, :p), NeuralPDE.var"#_RGF_ModTag", NeuralPDE.var"#_RGF_ModTag", (0x295b6845, 0xce1110b6, 0xa17beb73, 0xc449b733, 0xb3d6896e), Expr}, NeuralPDE.var"#12#13", NeuralPDE.var"#269#276"{NeuralPDE.var"#269#270#277"{typeof(NeuralPDE.numeric_derivative)}, Dict{Symbol, Int64}, Dict{Symbol, Int64}, GridTraining{Float64}}, typeof(NeuralPDE.numeric_derivative), Vector{NeuralPDE.Phi{Lux.Chain{NamedTuple{(:layer_1, :layer_2, :layer_3, :layer_4), Tuple{Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(identity), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}}}}, NamedTuple{(:layer_1, :layer_2, :layer_3, :layer_4), NTuple{4, NamedTuple{(), Tuple{}}}}}}, Vector{Float64}}, Matrix{Float64}}}, Vector{NeuralPDE.var"#74#75"{NeuralPDE.var"#209#210"{RuntimeGeneratedFunctions.RuntimeGeneratedFunction{(:cord, Symbol("##θ#293"), :phi, :derivative, :integral, :u, :p), NeuralPDE.var"#_RGF_ModTag", NeuralPDE.var"#_RGF_ModTag", (0xb5fb80bb, 0x61a62589, 0x2ec55669, 0x07bea97e, 0xb7ae3a52), Expr}, NeuralPDE.var"#12#13", NeuralPDE.var"#269#276"{NeuralPDE.var"#269#270#277"{typeof(NeuralPDE.numeric_derivative)}, Dict{Symbol, Int64}, Dict{Symbol, Int64}, GridTraining{Float64}}, typeof(NeuralPDE.numeric_derivative), Vector{NeuralPDE.Phi{Lux.Chain{NamedTuple{(:layer_1, :layer_2, :layer_3, :layer_4), Tuple{Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(identity), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}}}}, NamedTuple{(:layer_1, :layer_2, :layer_3, :layer_4), NTuple{4, NamedTuple{(), Tuple{}}}}}}, Vector{Float64}}, Matrix{Float64}}}, NeuralPDE.PINNRepresentation, Bool, Vector{Int64}, Int64, Vector{NeuralPDE.Phi{Lux.Chain{NamedTuple{(:layer_1, :layer_2, :layer_3, :layer_4), Tuple{Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(identity), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}}}}, NamedTuple{(:layer_1, :layer_2, :layer_3, :layer_4), NTuple{4, NamedTuple{(), Tuple{}}}}}}, Nothing, Bool, Vector{Float64}}, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, typeof(SciMLBase.DEFAULT_OBSERVED_NO_TIME), Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing}, Optimization.var"#228#237"{Optimization.var"#227#236"{OptimizationFunction{true, Optimization.AutoZygote, NeuralPDE.var"#full_loss_function#308"{NeuralPDE.var"#null_nonadaptive_loss#115", Vector{NeuralPDE.var"#74#75"{NeuralPDE.var"#209#210"{RuntimeGeneratedFunctions.RuntimeGeneratedFunction{(:cord, Symbol("##θ#293"), :phi, :derivative, :integral, :u, :p), NeuralPDE.var"#_RGF_ModTag", NeuralPDE.var"#_RGF_ModTag", (0x295b6845, 0xce1110b6, 0xa17beb73, 0xc449b733, 0xb3d6896e), Expr}, NeuralPDE.var"#12#13", NeuralPDE.var"#269#276"{NeuralPDE.var"#269#270#277"{typeof(NeuralPDE.numeric_derivative)}, Dict{Symbol, Int64}, Dict{Symbol, Int64}, GridTraining{Float64}}, typeof(NeuralPDE.numeric_derivative), Vector{NeuralPDE.Phi{Lux.Chain{NamedTuple{(:layer_1, :layer_2, :layer_3, :layer_4), Tuple{Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(identity), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}}}}, NamedTuple{(:layer_1, :layer_2, :layer_3, :layer_4), NTuple{4, NamedTuple{(), Tuple{}}}}}}, Vector{Float64}}, Matrix{Float64}}}, Vector{NeuralPDE.var"#74#75"{NeuralPDE.var"#209#210"{RuntimeGeneratedFunctions.RuntimeGeneratedFunction{(:cord, Symbol("##θ#293"), :phi, :derivative, :integral, :u, :p), NeuralPDE.var"#_RGF_ModTag", NeuralPDE.var"#_RGF_ModTag", (0xb5fb80bb, 0x61a62589, 0x2ec55669, 0x07bea97e, 0xb7ae3a52), Expr}, NeuralPDE.var"#12#13", NeuralPDE.var"#269#276"{NeuralPDE.var"#269#270#277"{typeof(NeuralPDE.numeric_derivative)}, Dict{Symbol, Int64}, Dict{Symbol, Int64}, GridTraining{Float64}}, typeof(NeuralPDE.numeric_derivative), Vector{NeuralPDE.Phi{Lux.Chain{NamedTuple{(:layer_1, :layer_2, :layer_3, :layer_4), Tuple{Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(identity), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}}}}, NamedTuple{(:layer_1, :layer_2, :layer_3, :layer_4), NTuple{4, NamedTuple{(), Tuple{}}}}}}, Vector{Float64}}, Matrix{Float64}}}, NeuralPDE.PINNRepresentation, Bool, Vector{Int64}, Int64, Vector{NeuralPDE.Phi{Lux.Chain{NamedTuple{(:layer_1, :layer_2, :layer_3, :layer_4), Tuple{Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(identity), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}}}}, NamedTuple{(:layer_1, :layer_2, :layer_3, :layer_4), NTuple{4, NamedTuple{(), Tuple{}}}}}}, Nothing, Bool, Vector{Float64}}, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, typeof(SciMLBase.DEFAULT_OBSERVED_NO_TIME), Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing}, Optimization.ReInitCache{ComponentArrays.ComponentVector{Float64, Vector{Float64}, Tuple{ComponentArrays.Axis{(depvar = ViewAxis(1:177, Axis(T_in = ViewAxis(1:177, Axis(layer_1 = ViewAxis(1:24, Axis(weight = ViewAxis(1:16, ShapedAxis((8, 2), NamedTuple())), bias = ViewAxis(17:24, ShapedAxis((8, 1), NamedTuple())))), layer_2 = ViewAxis(25:96, Axis(weight = ViewAxis(1:64, ShapedAxis((8, 8), NamedTuple())), bias = ViewAxis(65:72, ShapedAxis((8, 1), NamedTuple())))), layer_3 = ViewAxis(97:168, Axis(weight = ViewAxis(1:64, ShapedAxis((8, 8), NamedTuple())), bias = ViewAxis(65:72, ShapedAxis((8, 1), NamedTuple())))), layer_4 = ViewAxis(169:177, Axis(weight = ViewAxis(1:8, ShapedAxis((1, 8), NamedTuple())), bias = ViewAxis(9:9, ShapedAxis((1, 1), NamedTuple())))))),)), p = 178:180)}}}, SciMLBase.NullParameters}}}, Optimization.var"#231#240"{Optimization.var"#227#236"{OptimizationFunction{true, Optimization.AutoZygote, NeuralPDE.var"#full_loss_function#308"{NeuralPDE.var"#null_nonadaptive_loss#115", Vector{NeuralPDE.var"#74#75"{NeuralPDE.var"#209#210"{RuntimeGeneratedFunctions.RuntimeGeneratedFunction{(:cord, Symbol("##θ#293"), :phi, :derivative, :integral, :u, :p), NeuralPDE.var"#_RGF_ModTag", NeuralPDE.var"#_RGF_ModTag", (0x295b6845, 0xce1110b6, 0xa17beb73, 0xc449b733, 0xb3d6896e), Expr}, NeuralPDE.var"#12#13", NeuralPDE.var"#269#276"{NeuralPDE.var"#269#270#277"{typeof(NeuralPDE.numeric_derivative)}, Dict{Symbol, Int64}, Dict{Symbol, Int64}, GridTraining{Float64}}, typeof(NeuralPDE.numeric_derivative), Vector{NeuralPDE.Phi{Lux.Chain{NamedTuple{(:layer_1, :layer_2, :layer_3, :layer_4), Tuple{Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(identity), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}}}}, NamedTuple{(:layer_1, :layer_2, :layer_3, :layer_4), NTuple{4, NamedTuple{(), Tuple{}}}}}}, Vector{Float64}}, Matrix{Float64}}}, Vector{NeuralPDE.var"#74#75"{NeuralPDE.var"#209#210"{RuntimeGeneratedFunctions.RuntimeGeneratedFunction{(:cord, Symbol("##θ#293"), :phi, :derivative, :integral, :u, :p), NeuralPDE.var"#_RGF_ModTag", NeuralPDE.var"#_RGF_ModTag", (0xb5fb80bb, 0x61a62589, 0x2ec55669, 0x07bea97e, 0xb7ae3a52), Expr}, NeuralPDE.var"#12#13", NeuralPDE.var"#269#276"{NeuralPDE.var"#269#270#277"{typeof(NeuralPDE.numeric_derivative)}, Dict{Symbol, Int64}, Dict{Symbol, Int64}, GridTraining{Float64}}, typeof(NeuralPDE.numeric_derivative), Vector{NeuralPDE.Phi{Lux.Chain{NamedTuple{(:layer_1, :layer_2, :layer_3, :layer_4), Tuple{Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(identity), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}}}}, NamedTuple{(:layer_1, :layer_2, :layer_3, :layer_4), NTuple{4, NamedTuple{(), Tuple{}}}}}}, Vector{Float64}}, Matrix{Float64}}}, NeuralPDE.PINNRepresentation, Bool, Vector{Int64}, Int64, Vector{NeuralPDE.Phi{Lux.Chain{NamedTuple{(:layer_1, :layer_2, :layer_3, :layer_4), Tuple{Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(identity), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}}}}, NamedTuple{(:layer_1, :layer_2, :layer_3, :layer_4), NTuple{4, NamedTuple{(), Tuple{}}}}}}, Nothing, Bool, Vector{Float64}}, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, typeof(SciMLBase.DEFAULT_OBSERVED_NO_TIME), Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing}, Optimization.ReInitCache{ComponentArrays.ComponentVector{Float64, Vector{Float64}, Tuple{ComponentArrays.Axis{(depvar = ViewAxis(1:177, Axis(T_in = ViewAxis(1:177, Axis(layer_1 = ViewAxis(1:24, Axis(weight = ViewAxis(1:16, ShapedAxis((8, 2), NamedTuple())), bias = ViewAxis(17:24, ShapedAxis((8, 1), NamedTuple())))), layer_2 = ViewAxis(25:96, Axis(weight = ViewAxis(1:64, ShapedAxis((8, 8), NamedTuple())), bias = ViewAxis(65:72, ShapedAxis((8, 1), NamedTuple())))), layer_3 = ViewAxis(97:168, Axis(weight = ViewAxis(1:64, ShapedAxis((8, 8), NamedTuple())), bias = ViewAxis(65:72, ShapedAxis((8, 1), NamedTuple())))), layer_4 = ViewAxis(169:177, Axis(weight = ViewAxis(1:8, ShapedAxis((1, 8), NamedTuple())), bias = ViewAxis(9:9, ShapedAxis((1, 1), NamedTuple())))))),)), p = 178:180)}}}, SciMLBase.NullParameters}}}, Optimization.var"#235#244", Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, typeof(SciMLBase.DEFAULT_OBSERVED_NO_TIME), Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing}, Optimization.ReInitCache{ComponentArrays.ComponentVector{Float64, Vector{Float64}, Tuple{ComponentArrays.Axis{(depvar = ViewAxis(1:177, Axis(T_in = ViewAxis(1:177, Axis(layer_1 = ViewAxis(1:24, Axis(weight = ViewAxis(1:16, ShapedAxis((8, 2), NamedTuple())), bias = ViewAxis(17:24, ShapedAxis((8, 1), NamedTuple())))), layer_2 = ViewAxis(25:96, Axis(weight = ViewAxis(1:64, ShapedAxis((8, 8), NamedTuple())), bias = ViewAxis(65:72, ShapedAxis((8, 1), NamedTuple())))), layer_3 = ViewAxis(97:168, Axis(weight = ViewAxis(1:64, ShapedAxis((8, 8), NamedTuple())), bias = ViewAxis(65:72, ShapedAxis((8, 1), NamedTuple())))), layer_4 = ViewAxis(169:177, Axis(weight = ViewAxis(1:8, ShapedAxis((1, 8), NamedTuple())), bias = ViewAxis(9:9, ShapedAxis((1, 1), NamedTuple())))))),)), p = 178:180)}}}, SciMLBase.NullParameters}, Nothing, Nothing, Nothing, Nothing, Nothing, BFGS{LineSearches.InitialStatic{Float64}, LineSearches.HagerZhang{Float64, Base.RefValue{Bool}}, Nothing, Nothing, Flat}, Base.Iterators.Cycle{Tuple{Optimization.NullData}}, Bool, var"#9#10"})
    @ SciMLBase C:\Users\loren\.julia\packages\SciMLBase\QqtZA\src\solve.jl:160
 [70] solve(::OptimizationProblem{true, OptimizationFunction{true, Optimization.AutoZygote, NeuralPDE.var"#full_loss_function#308"{NeuralPDE.var"#null_nonadaptive_loss#115", Vector{NeuralPDE.var"#74#75"{NeuralPDE.var"#209#210"{RuntimeGeneratedFunctions.RuntimeGeneratedFunction{(:cord, Symbol("##θ#293"), :phi, :derivative, :integral, :u, :p), NeuralPDE.var"#_RGF_ModTag", NeuralPDE.var"#_RGF_ModTag", (0x295b6845, 0xce1110b6, 0xa17beb73, 0xc449b733, 0xb3d6896e), Expr}, NeuralPDE.var"#12#13", NeuralPDE.var"#269#276"{NeuralPDE.var"#269#270#277"{typeof(NeuralPDE.numeric_derivative)}, Dict{Symbol, Int64}, Dict{Symbol, Int64}, GridTraining{Float64}}, typeof(NeuralPDE.numeric_derivative), Vector{NeuralPDE.Phi{Lux.Chain{NamedTuple{(:layer_1, :layer_2, :layer_3, :layer_4), Tuple{Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(identity), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}}}}, NamedTuple{(:layer_1, :layer_2, :layer_3, :layer_4), NTuple{4, NamedTuple{(), Tuple{}}}}}}, Vector{Float64}}, Matrix{Float64}}}, Vector{NeuralPDE.var"#74#75"{NeuralPDE.var"#209#210"{RuntimeGeneratedFunctions.RuntimeGeneratedFunction{(:cord, Symbol("##θ#293"), :phi, :derivative, :integral, :u, :p), NeuralPDE.var"#_RGF_ModTag", NeuralPDE.var"#_RGF_ModTag", (0xb5fb80bb, 0x61a62589, 0x2ec55669, 0x07bea97e, 0xb7ae3a52), Expr}, NeuralPDE.var"#12#13", NeuralPDE.var"#269#276"{NeuralPDE.var"#269#270#277"{typeof(NeuralPDE.numeric_derivative)}, Dict{Symbol, Int64}, Dict{Symbol, Int64}, GridTraining{Float64}}, typeof(NeuralPDE.numeric_derivative), Vector{NeuralPDE.Phi{Lux.Chain{NamedTuple{(:layer_1, :layer_2, :layer_3, :layer_4), Tuple{Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(identity), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}}}}, NamedTuple{(:layer_1, :layer_2, :layer_3, :layer_4), NTuple{4, NamedTuple{(), Tuple{}}}}}}, Vector{Float64}}, Matrix{Float64}}}, NeuralPDE.PINNRepresentation, Bool, Vector{Int64}, Int64, Vector{NeuralPDE.Phi{Lux.Chain{NamedTuple{(:layer_1, :layer_2, :layer_3, :layer_4), Tuple{Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(identity), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}}}}, NamedTuple{(:layer_1, :layer_2, :layer_3, :layer_4), NTuple{4, NamedTuple{(), Tuple{}}}}}}, Nothing, Bool, Vector{Float64}}, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, typeof(SciMLBase.DEFAULT_OBSERVED_NO_TIME), Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing}, ComponentArrays.ComponentVector{Float64, Vector{Float64}, Tuple{ComponentArrays.Axis{(depvar = ViewAxis(1:177, Axis(T_in = ViewAxis(1:177, Axis(layer_1 = ViewAxis(1:24, Axis(weight = ViewAxis(1:16, ShapedAxis((8, 2), NamedTuple())), bias = ViewAxis(17:24, ShapedAxis((8, 1), NamedTuple())))), layer_2 = ViewAxis(25:96, Axis(weight = ViewAxis(1:64, ShapedAxis((8, 8), NamedTuple())), bias = ViewAxis(65:72, ShapedAxis((8, 1), NamedTuple())))), layer_3 = ViewAxis(97:168, Axis(weight = ViewAxis(1:64, ShapedAxis((8, 8), NamedTuple())), bias = ViewAxis(65:72, ShapedAxis((8, 1), NamedTuple())))), layer_4 = ViewAxis(169:177, Axis(weight = ViewAxis(1:8, ShapedAxis((1, 8), NamedTuple())), bias = ViewAxis(9:9, ShapedAxis((1, 1), NamedTuple())))))),)), p = 178:180)}}}, SciMLBase.NullParameters, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Base.Pairs{Symbol, Union{}, Tuple{}, NamedTuple{(), Tuple{}}}}, ::BFGS{LineSearches.InitialStatic{Float64}, LineSearches.HagerZhang{Float64, Base.RefValue{Bool}}, Nothing, Nothing, Flat}; kwargs::Base.Pairs{Symbol, Any, Tuple{Symbol, Symbol}, NamedTuple{(:maxiters, :callback), Tuple{Int64, var"#9#10"}}})
    @ SciMLBase C:\Users\loren\.julia\packages\SciMLBase\QqtZA\src\solve.jl:8

Astitva Aggarwal · Answer 8 · Thu Nov 02 2023 17:52:20 GMT+0800 (China Standard Time)

using NeuralPDE, Lux, ModelingToolkit, DataFrames, CSV, DataLoaders, Flux, IntervalSets, Optimization, OptimizationOptimJL

# Definisci il modello dell'equazione differenziale
@variables t::Real

@syms T_in(t)::Real
@syms T_ext(t)::Real
@syms Q_heating(t)::Real
@syms Q_cooling(t)::Real
@syms Q_sun(t)::Real
@syms Q_lights(t)::Real
@syms Q_equipment(t)::Real
@parameters R::Real C::Real Cs::Real

#details of problem to be solved
Dt = Differential(t)
eqs = [Dt(T_in(t)) ~ (T_ext(t) - T_in(t)) / (R * C) + Q_heating(t) / C - Q_cooling(t) / C + Q_sun(t) / Cs + (Q_lights(t) + Q_equipment(t)) / C]
tspan = (0.0, 365.0 * 24.0 * 60.0)  # Dati per un anno
domains = [t ∈ (0.0,365.0 * 24.0 * 60.0)]
bcs=[Dt(T_in(0))~19.3]

# dataset creation and additional loss function
data = CSV.File("shoebox_free.csv") |> DataFrame

T_ext_data = data."Environment:Site Outdoor Air Drybulb Temperature [C](TimeStep)"
Q_heating_data = data."OSGB1000005735772_FLOOR_1_HVAC:Zone Ideal Loads Zone Total Heating Rate [W](TimeStep)"
Q_cooling_data = data."OSGB1000005735772_FLOOR_1_HVAC:Zone Ideal Loads Zone Total Cooling Rate [W](TimeStep)"
Q_sun_data = data."Environment:Site Direct Solar Radiation Rate per Area [W/m2](TimeStep)"
Q_lights_data = data."OSGB1000005735772_FLOOR_1:Zone Lights Total Heating Rate [W](TimeStep)"
Q_equipment_data = data."OSGB1000005735772_FLOOR_1:Zone Electric Equipment Total Heating Rate [W](TimeStep)"

t_data = 1:size(data, 1)

dataloader = DataLoader([vcat(T_ext_data, Q_heating_data, Q_cooling_data, Q_sun_data, Q_lights_data, Q_equipment_data, t_data)])

function additional_loss(phi, θ, p)
    T_in_predict = phi[1](t_data, θ[1])[1]
    return sum(abs2(T_in_predict .- T_in_data) / length(data))
end

# Creating chain
input_dim = length(tspan)
hidden_units = 8
chain1 = Lux.Chain(Lux.Dense(input_dim, hidden_units, Lux.σ),
                   Lux.Dense(hidden_units, hidden_units, Lux.σ),
                   Lux.Dense(hidden_units, hidden_units, Lux.σ),
                   Lux.Dense(hidden_units, 1))

# discretize domain with PINN
dt = 600.0  # 600 secondi (10 minuti)
discretization = NeuralPDE.PhysicsInformedNN([chain1],
                                             NeuralPDE.GridTraining(dt),
                                             param_estim = true, additional_loss = additional_loss)

@named pde_system=NeuralPDE.PDESystem(eqs,bcs,domains, [t], [T_in(t), T_ext(t), Q_heating(t), Q_cooling(t), Q_sun(t), Q_lights(t), Q_equipment(t)], [R, C, Cs], defaults = Dict([R => 1.0, C => 1.0, Cs => 1.0]))

prob = NeuralPDE.discretize(pde_system, discretization)

# solve
res = Optimization.solve(prob, BFGS(), maxiters = 5000, callback = (p, l) -> println("Current loss is: $l"))

# checking solution
p_optimized = res.u[end]

minimizer = res.u.depvar[1]
T_in_predict = minimizer(t_data)

using Plots
plot(t_data, T_in_data, label = "Dati Osservati")
plot!(t_data, T_in_predict, label = "Temperatura Prevista", linestyle = :dash)

@LorenzoBottaccioli try using this(ive just passed the additional loss funciton in PhysicsInformedNN() and formatted the code), the error is mostly due to some wrong argument being passed

LorenzoBottaccioli · Answer 9 · Thu Nov 02 2023 18:21:58 GMT+0800 (China Standard Time)

@AstitvaAggarwal thanks for your help. I dont get how I could solve the error of type NamedTuple has no filed T_ext. Here if you want there is the input data
shoebox_free.csv

Astitva Aggarwal · Answer 10 · Thu Nov 02 2023 22:26:13 GMT+0800 (China Standard Time)

@LorenzoBottaccioli You are mostly defining the additional loss function incorrectly. (something like below from tutorial)

depvars = [:x, :y, :z]
function additional_loss(phi, θ, p)
    return sum(sum(abs2, phi[i](t_, θ[depvars[i]]) .- u_[[i], :]) / len for i in 1:1:3)
end

also Q_heating_data etc after T_ext_data are all zero datavectors.

LorenzoBottaccioli · Answer 11 · Fri Nov 03 2023 00:09:28 GMT+0800 (China Standard Time)

@AstitvaAggarwal still I'm not understanding the tutorial at least for me it is not very clear and explanatory. The other data are zeros I know. The case study is shoebox building that is running in free-float, hence no heating or cooling so far and no people inside, just solar radiation as gain. What I'm not getting in general is how I pass the data from T_et of any other feature to the PINN problem.

LorenzoBottaccioli · Answer 12 · Thu Nov 09 2023 23:08:34 GMT+0800 (China Standard Time)

@AstitvaAggarwal @sathvikbhagavan I have re-arranged the code in a more comprehensive way but I get a new error. Can you help me in solving it?

using NeuralPDE, Lux, ModelingToolkit, DataFrames, CSV, DataLoaders, Flux, IntervalSets, Optimization, OptimizationOptimJL, DataInterpolations

# Definisci il modello dell'equazione differenziale
#@variables t::Real

@syms T_in(t)::Real
@syms T_ext(t)::Real
@syms Q_heating(t)::Real
@syms Q_cooling(t)::Real
@syms Q_sun(t)::Real
@syms Q_lights(t)::Real
@syms Q_equipment(t)::Real
@parameters R, C, Cs, t


Dt = Differential(t)

eqs = [Dt(T_in(t)) ~ (-T_ext(t) + T_in(t)) / (R * C) + Q_heating(t) / C - Q_cooling(t) / C + Q_sun(t) / Cs + (Q_lights(t) + Q_equipment(t)) / C]

domains = [t ∈ (0.0,365.0 * 24.0 * 60.0)]
bcs=[Dt(T_in(0))~4.48]
dt = 600.0  # 600 seconds (10 minute)

# Define the temporal space
tspan = (0.0, 365.0 * 24.0 * 60.0)  # Dati per un anno

# load sampled data from CSV
data = CSV.File("shoebox_free.csv") |> DataFrame

# Put the sampled data in dedicated variables
T_in_data = data."OSGB1000005735772_FLOOR_1:Zone Mean Air Temperature [C](TimeStep)"
T_ext_data = data."Environment:Site Outdoor Air Drybulb Temperature [C](TimeStep)"
Q_heating_data = data."OSGB1000005735772_FLOOR_1_HVAC:Zone Ideal Loads Zone Total Heating Rate [W](TimeStep)"
Q_cooling_data = data."OSGB1000005735772_FLOOR_1_HVAC:Zone Ideal Loads Zone Total Cooling Rate [W](TimeStep)"
Q_sun_data = data."Environment:Site Direct Solar Radiation Rate per Area [W/m2](TimeStep)"
Q_lights_data = data."OSGB1000005735772_FLOOR_1:Zone Lights Total Heating Rate [W](TimeStep)"
Q_equipment_data = data."OSGB1000005735772_FLOOR_1:Zone Electric Equipment Total Heating Rate [W](TimeStep)"

t_data = 1:size(data, 1)

dataloader = DataLoader([vcat(T_in_data,T_ext_data, Q_heating_data, Q_cooling_data, Q_sun_data, Q_lights_data, Q_equipment_data, t_data)])

# Define the NN
input_dim = 7
hidden_units = 8
len=length(t_data)

chain1 = Lux.Chain(Lux.Dense(input_dim, hidden_units, Lux.σ),
                   Lux.Dense(hidden_units, hidden_units, Lux.σ),
                   Lux.Dense(hidden_units, hidden_units, Lux.σ),
                   Lux.Dense(hidden_units, 1))

#Define dependent and independent vatiables
indvars=[t,T_ext,Q_heating,Q_cooling,Q_sun,Q_lights,Q_equipment]
depvars = [T_in]
# Define the loss
function additional_loss(phi, θ, p)
    T_in_predict = phi(t_data, θ[depvars])
    return sum(abs2(T_in_predict .- T_in_data))# / len)
end

@named pde_system=NeuralPDE.PDESystem(eqs,bcs,domains,indvars,depvars, [R, C, Cs], defaults = Dict([R => 1.0, C => 1.0, Cs => 1.0]))#[R, C, Cs])


discretization = NeuralPDE.PhysicsInformedNN(chain1,
                                             NeuralPDE.GridTraining(dt),
                                             param_estim = true, additional_loss = additional_loss)
prob = NeuralPDE.discretize(pde_system, discretization)

# Parameter Optimization
res = Optimization.solve(prob, BFGS(), maxiters = 5000, callback = (p, l) -> println("Current loss is: $l"))

p_optimized = res.u[end]
# Plot fo results
minimizer = res.u.depvar[1]
T_in_predict = minimizer(t_data)

using Plots
plot(t_data, T_in_data, label = "Dati Osservati")
plot!(t_data, T_in_predict, label = "Temperatura Prevista", linestyle = :dash)

I get the following error. What I still don't understand is when and how I should use the data I have loaded.

BoundsError: attempt to access 1×877 Matrix{Float64} at index [[2], 1:877]

Stacktrace:
  [1] throw_boundserror(A::Matrix{Float64}, I::Tuple{Vector{Int64}, Base.Slice{Base.OneTo{Int64}}})
    @ Base .\abstractarray.jl:744
  [2] checkbounds
    @ .\abstractarray.jl:709 [inlined]
  [3] _getindex
    @ .\multidimensional.jl:860 [inlined]
  [4] getindex
    @ .\abstractarray.jl:1294 [inlined]
  [5] rrule(::typeof(getindex), ::Matrix{Float64}, ::Vector{Int64}, ::Function)
    @ ChainRules C:\Users\loren\.julia\packages\ChainRules\snrkz\src\rulesets\Base\indexing.jl:63
  [6] rrule
    @ C:\Users\loren\.julia\packages\ChainRulesCore\7MWx2\src\rules.jl:134 [inlined]
  [7] chain_rrule
    @ C:\Users\loren\.julia\packages\Zygote\YYT6v\src\compiler\chainrules.jl:223 [inlined]
  [8] macro expansion
    @ C:\Users\loren\.julia\packages\Zygote\YYT6v\src\compiler\interface2.jl:101 [inlined]
  [9] _pullback
    @ C:\Users\loren\.julia\packages\Zygote\YYT6v\src\compiler\interface2.jl:101 [inlined]
 [10] macro expansion
    @ C:\Users\loren\.julia\packages\NeuralPDE\F4RlZ\src\discretize.jl:169 [inlined]
 [11] macro expansion
    @ C:\Users\loren\.julia\packages\RuntimeGeneratedFunctions\Yo8zx\src\RuntimeGeneratedFunctions.jl:163 [inlined]
 [12] macro expansion
    @ .\none:0 [inlined]
 [13] _pullback
    @ .\none:0 [inlined]
 [14] _pullback(::Zygote.Context{false}, ::typeof(RuntimeGeneratedFunctions.generated_callfunc), ::RuntimeGeneratedFunctions.RuntimeGeneratedFunction{(:cord, Symbol("##θ#293"), :phi, :derivative, :integral, :u, :p), NeuralPDE.var"#_RGF_ModTag", NeuralPDE.var"#_RGF_ModTag", (0x7cf59a19, 0x0162d753, 0x260a99e5, 0xc4bc113e, 0x070b6155), Expr}, ::Matrix{Float64}, ::ComponentArrays.ComponentVector{Float64, Vector{Float64}, Tuple{ComponentArrays.Axis{(depvar = ViewAxis(1:217, Axis(layer_1 = ViewAxis(1:64, Axis(weight = ViewAxis(1:56, ShapedAxis((8, 7), NamedTuple())), bias = ViewAxis(57:64, ShapedAxis((8, 1), NamedTuple())))), layer_2 = ViewAxis(65:136, Axis(weight = ViewAxis(1:64, ShapedAxis((8, 8), NamedTuple())), bias = ViewAxis(65:72, ShapedAxis((8, 1), NamedTuple())))), layer_3 = ViewAxis(137:208, Axis(weight = ViewAxis(1:64, ShapedAxis((8, 8), NamedTuple())), bias = ViewAxis(65:72, ShapedAxis((8, 1), NamedTuple())))), layer_4 = ViewAxis(209:217, Axis(weight = ViewAxis(1:8, ShapedAxis((1, 8), NamedTuple())), bias = ViewAxis(9:9, ShapedAxis((1, 1), NamedTuple())))))), p = 218:220)}}}, ::NeuralPDE.Phi{Lux.Chain{NamedTuple{(:layer_1, :layer_2, :layer_3, :layer_4), Tuple{Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(identity), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}}}}, NamedTuple{(:layer_1, :layer_2, :layer_3, :layer_4), NTuple{4, NamedTuple{(), Tuple{}}}}}, ::typeof(NeuralPDE.numeric_derivative), ::NeuralPDE.var"#269#276"{NeuralPDE.var"#269#270#277"{typeof(NeuralPDE.numeric_derivative)}, Dict{Symbol, Int64}, Dict{Symbol, Int64}, GridTraining{Float64}}, ::NeuralPDE.var"#12#13", ::Vector{Float64})
    @ Zygote C:\Users\loren\.julia\packages\Zygote\YYT6v\src\compiler\interface2.jl:0
 [15] _apply(::Function, ::Vararg{Any})
    @ Core .\boot.jl:838
 [16] adjoint
    @ C:\Users\loren\.julia\packages\Zygote\YYT6v\src\lib\lib.jl:203 [inlined]
 [17] _pullback
    @ C:\Users\loren\.julia\packages\ZygoteRules\4nXuu\src\adjoint.jl:66 [inlined]
 [18] _pullback
    @ C:\Users\loren\.julia\packages\RuntimeGeneratedFunctions\Yo8zx\src\RuntimeGeneratedFunctions.jl:150 [inlined]
 [19] _pullback(::Zygote.Context{false}, ::RuntimeGeneratedFunctions.RuntimeGeneratedFunction{(:cord, Symbol("##θ#293"), :phi, :derivative, :integral, :u, :p), NeuralPDE.var"#_RGF_ModTag", NeuralPDE.var"#_RGF_ModTag", (0x7cf59a19, 0x0162d753, 0x260a99e5, 0xc4bc113e, 0x070b6155), Expr}, ::Matrix{Float64}, ::ComponentArrays.ComponentVector{Float64, Vector{Float64}, Tuple{ComponentArrays.Axis{(depvar = ViewAxis(1:217, Axis(layer_1 = ViewAxis(1:64, Axis(weight = ViewAxis(1:56, ShapedAxis((8, 7), NamedTuple())), bias = ViewAxis(57:64, ShapedAxis((8, 1), NamedTuple())))), layer_2 = ViewAxis(65:136, Axis(weight = ViewAxis(1:64, ShapedAxis((8, 8), NamedTuple())), bias = ViewAxis(65:72, ShapedAxis((8, 1), NamedTuple())))), layer_3 = ViewAxis(137:208, Axis(weight = ViewAxis(1:64, ShapedAxis((8, 8), NamedTuple())), bias = ViewAxis(65:72, ShapedAxis((8, 1), NamedTuple())))), layer_4 = ViewAxis(209:217, Axis(weight = ViewAxis(1:8, ShapedAxis((1, 8), NamedTuple())), bias = ViewAxis(9:9, ShapedAxis((1, 1), NamedTuple())))))), p = 218:220)}}}, ::NeuralPDE.Phi{Lux.Chain{NamedTuple{(:layer_1, :layer_2, :layer_3, :layer_4), Tuple{Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(identity), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}}}}, NamedTuple{(:layer_1, :layer_2, :layer_3, :layer_4), NTuple{4, NamedTuple{(), Tuple{}}}}}, ::typeof(NeuralPDE.numeric_derivative), ::NeuralPDE.var"#269#276"{NeuralPDE.var"#269#270#277"{typeof(NeuralPDE.numeric_derivative)}, Dict{Symbol, Int64}, Dict{Symbol, Int64}, GridTraining{Float64}}, ::NeuralPDE.var"#12#13", ::Vector{Float64})
    @ Zygote C:\Users\loren\.julia\packages\Zygote\YYT6v\src\compiler\interface2.jl:0
 [20] _pullback
    @ C:\Users\loren\.julia\packages\NeuralPDE\F4RlZ\src\discretize.jl:192 [inlined]
 [21] _pullback(::Zygote.Context{false}, ::NeuralPDE.var"#209#210"{RuntimeGeneratedFunctions.RuntimeGeneratedFunction{(:cord, Symbol("##θ#293"), :phi, :derivative, :integral, :u, :p), NeuralPDE.var"#_RGF_ModTag", NeuralPDE.var"#_RGF_ModTag", (0x7cf59a19, 0x0162d753, 0x260a99e5, 0xc4bc113e, 0x070b6155), Expr}, NeuralPDE.var"#12#13", NeuralPDE.var"#269#276"{NeuralPDE.var"#269#270#277"{typeof(NeuralPDE.numeric_derivative)}, Dict{Symbol, Int64}, Dict{Symbol, Int64}, GridTraining{Float64}}, typeof(NeuralPDE.numeric_derivative), NeuralPDE.Phi{Lux.Chain{NamedTuple{(:layer_1, :layer_2, :layer_3, :layer_4), Tuple{Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(identity), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}}}}, NamedTuple{(:layer_1, :layer_2, :layer_3, :layer_4), NTuple{4, NamedTuple{(), Tuple{}}}}}, Vector{Float64}}, ::Matrix{Float64}, ::ComponentArrays.ComponentVector{Float64, Vector{Float64}, Tuple{ComponentArrays.Axis{(depvar = ViewAxis(1:217, Axis(layer_1 = ViewAxis(1:64, Axis(weight = ViewAxis(1:56, ShapedAxis((8, 7), NamedTuple())), bias = ViewAxis(57:64, ShapedAxis((8, 1), NamedTuple())))), layer_2 = ViewAxis(65:136, Axis(weight = ViewAxis(1:64, ShapedAxis((8, 8), NamedTuple())), bias = ViewAxis(65:72, ShapedAxis((8, 1), NamedTuple())))), layer_3 = ViewAxis(137:208, Axis(weight = ViewAxis(1:64, ShapedAxis((8, 8), NamedTuple())), bias = ViewAxis(65:72, ShapedAxis((8, 1), NamedTuple())))), layer_4 = ViewAxis(209:217, Axis(weight = ViewAxis(1:8, ShapedAxis((1, 8), NamedTuple())), bias = ViewAxis(9:9, ShapedAxis((1, 1), NamedTuple())))))), p = 218:220)}}})
    @ Zygote C:\Users\loren\.julia\packages\Zygote\YYT6v\src\compiler\interface2.jl:0
 [22] _pullback
    @ C:\Users\loren\.julia\packages\NeuralPDE\F4RlZ\src\training_strategies.jl:48 [inlined]
 [23] _pullback(ctx::Zygote.Context{false}, f::NeuralPDE.var"#74#75"{NeuralPDE.var"#209#210"{RuntimeGeneratedFunctions.RuntimeGeneratedFunction{(:cord, Symbol("##θ#293"), :phi, :derivative, :integral, :u, :p), NeuralPDE.var"#_RGF_ModTag", NeuralPDE.var"#_RGF_ModTag", (0x7cf59a19, 0x0162d753, 0x260a99e5, 0xc4bc113e, 0x070b6155), Expr}, NeuralPDE.var"#12#13", NeuralPDE.var"#269#276"{NeuralPDE.var"#269#270#277"{typeof(NeuralPDE.numeric_derivative)}, Dict{Symbol, Int64}, Dict{Symbol, Int64}, GridTraining{Float64}}, typeof(NeuralPDE.numeric_derivative), NeuralPDE.Phi{Lux.Chain{NamedTuple{(:layer_1, :layer_2, :layer_3, :layer_4), Tuple{Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(identity), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}}}}, NamedTuple{(:layer_1, :layer_2, :layer_3, :layer_4), NTuple{4, NamedTuple{(), Tuple{}}}}}, Vector{Float64}}, Matrix{Float64}}, args::ComponentArrays.ComponentVector{Float64, Vector{Float64}, Tuple{ComponentArrays.Axis{(depvar = ViewAxis(1:217, Axis(layer_1 = ViewAxis(1:64, Axis(weight = ViewAxis(1:56, ShapedAxis((8, 7), NamedTuple())), bias = ViewAxis(57:64, ShapedAxis((8, 1), NamedTuple())))), layer_2 = ViewAxis(65:136, Axis(weight = ViewAxis(1:64, ShapedAxis((8, 8), NamedTuple())), bias = ViewAxis(65:72, ShapedAxis((8, 1), NamedTuple())))), layer_3 = ViewAxis(137:208, Axis(weight = ViewAxis(1:64, ShapedAxis((8, 8), NamedTuple())), bias = ViewAxis(65:72, ShapedAxis((8, 1), NamedTuple())))), layer_4 = ViewAxis(209:217, Axis(weight = ViewAxis(1:8, ShapedAxis((1, 8), NamedTuple())), bias = ViewAxis(9:9, ShapedAxis((1, 1), NamedTuple())))))), p = 218:220)}}})
    @ Zygote C:\Users\loren\.julia\packages\Zygote\YYT6v\src\compiler\interface2.jl:0
 [24] _pullback
    @ .\none:0 [inlined]
 [25] _pullback(ctx::Zygote.Context{false}, f::NeuralPDE.var"#294#309"{ComponentArrays.ComponentVector{Float64, Vector{Float64}, Tuple{ComponentArrays.Axis{(depvar = ViewAxis(1:217, Axis(layer_1 = ViewAxis(1:64, Axis(weight = ViewAxis(1:56, ShapedAxis((8, 7), NamedTuple())), bias = ViewAxis(57:64, ShapedAxis((8, 1), NamedTuple())))), layer_2 = ViewAxis(65:136, Axis(weight = ViewAxis(1:64, ShapedAxis((8, 8), NamedTuple())), bias = ViewAxis(65:72, ShapedAxis((8, 1), NamedTuple())))), layer_3 = ViewAxis(137:208, Axis(weight = ViewAxis(1:64, ShapedAxis((8, 8), NamedTuple())), bias = ViewAxis(65:72, ShapedAxis((8, 1), NamedTuple())))), layer_4 = ViewAxis(209:217, Axis(weight = ViewAxis(1:8, ShapedAxis((1, 8), NamedTuple())), bias = ViewAxis(9:9, ShapedAxis((1, 1), NamedTuple())))))), p = 218:220)}}}}, args::NeuralPDE.var"#74#75"{NeuralPDE.var"#209#210"{RuntimeGeneratedFunctions.RuntimeGeneratedFunction{(:cord, Symbol("##θ#293"), :phi, :derivative, :integral, :u, :p), NeuralPDE.var"#_RGF_ModTag", NeuralPDE.var"#_RGF_ModTag", (0x7cf59a19, 0x0162d753, 0x260a99e5, 0xc4bc113e, 0x070b6155), Expr}, NeuralPDE.var"#12#13", NeuralPDE.var"#269#276"{NeuralPDE.var"#269#270#277"{typeof(NeuralPDE.numeric_derivative)}, Dict{Symbol, Int64}, Dict{Symbol, Int64}, GridTraining{Float64}}, typeof(NeuralPDE.numeric_derivative), NeuralPDE.Phi{Lux.Chain{NamedTuple{(:layer_1, :layer_2, :layer_3, :layer_4), Tuple{Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(identity), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}}}}, NamedTuple{(:layer_1, :layer_2, :layer_3, :layer_4), NTuple{4, NamedTuple{(), Tuple{}}}}}, Vector{Float64}}, Matrix{Float64}})
    @ Zygote C:\Users\loren\.julia\packages\Zygote\YYT6v\src\compiler\interface2.jl:0
 [26] #661
    @ C:\Users\loren\.julia\packages\Zygote\YYT6v\src\lib\array.jl:183 [inlined]
 [27] iterate
    @ .\generator.jl:47 [inlined]
 [28] _collect(c::Vector{NeuralPDE.var"#74#75"{NeuralPDE.var"#209#210"{RuntimeGeneratedFunctions.RuntimeGeneratedFunction{(:cord, Symbol("##θ#293"), :phi, :derivative, :integral, :u, :p), NeuralPDE.var"#_RGF_ModTag", NeuralPDE.var"#_RGF_ModTag", (0x7cf59a19, 0x0162d753, 0x260a99e5, 0xc4bc113e, 0x070b6155), Expr}, NeuralPDE.var"#12#13", NeuralPDE.var"#269#276"{NeuralPDE.var"#269#270#277"{typeof(NeuralPDE.numeric_derivative)}, Dict{Symbol, Int64}, Dict{Symbol, Int64}, GridTraining{Float64}}, typeof(NeuralPDE.numeric_derivative), NeuralPDE.Phi{Lux.Chain{NamedTuple{(:layer_1, :layer_2, :layer_3, :layer_4), Tuple{Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(identity), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}}}}, NamedTuple{(:layer_1, :layer_2, :layer_3, :layer_4), NTuple{4, NamedTuple{(), Tuple{}}}}}, Vector{Float64}}, Matrix{Float64}}}, itr::Base.Generator{Vector{NeuralPDE.var"#74#75"{NeuralPDE.var"#209#210"{RuntimeGeneratedFunctions.RuntimeGeneratedFunction{(:cord, Symbol("##θ#293"), :phi, :derivative, :integral, :u, :p), NeuralPDE.var"#_RGF_ModTag", NeuralPDE.var"#_RGF_ModTag", (0x7cf59a19, 0x0162d753, 0x260a99e5, 0xc4bc113e, 0x070b6155), Expr}, NeuralPDE.var"#12#13", NeuralPDE.var"#269#276"{NeuralPDE.var"#269#270#277"{typeof(NeuralPDE.numeric_derivative)}, Dict{Symbol, Int64}, Dict{Symbol, Int64}, GridTraining{Float64}}, typeof(NeuralPDE.numeric_derivative), NeuralPDE.Phi{Lux.Chain{NamedTuple{(:layer_1, :layer_2, :layer_3, :layer_4), Tuple{Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(identity), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}}}}, NamedTuple{(:layer_1, :layer_2, :layer_3, :layer_4), NTuple{4, NamedTuple{(), Tuple{}}}}}, Vector{Float64}}, Matrix{Float64}}}, Zygote.var"#661#665"{Zygote.Context{false}, NeuralPDE.var"#294#309"{ComponentArrays.ComponentVector{Float64, Vector{Float64}, Tuple{ComponentArrays.Axis{(depvar = ViewAxis(1:217, Axis(layer_1 = ViewAxis(1:64, Axis(weight = ViewAxis(1:56, ShapedAxis((8, 7), NamedTuple())), bias = ViewAxis(57:64, ShapedAxis((8, 1), NamedTuple())))), layer_2 = ViewAxis(65:136, Axis(weight = ViewAxis(1:64, ShapedAxis((8, 8), NamedTuple())), bias = ViewAxis(65:72, ShapedAxis((8, 1), NamedTuple())))), layer_3 = ViewAxis(137:208, Axis(weight = ViewAxis(1:64, ShapedAxis((8, 8), NamedTuple())), bias = ViewAxis(65:72, ShapedAxis((8, 1), NamedTuple())))), layer_4 = ViewAxis(209:217, Axis(weight = ViewAxis(1:8, ShapedAxis((1, 8), NamedTuple())), bias = ViewAxis(9:9, ShapedAxis((1, 1), NamedTuple())))))), p = 218:220)}}}}}}, #unused#::Base.EltypeUnknown, isz::Base.HasShape{1})
    @ Base .\array.jl:802
 [29] collect_similar
    @ .\array.jl:711 [inlined]
 [30] map
    @ .\abstractarray.jl:3261 [inlined]
 [31] ∇map(cx::Zygote.Context{false}, f::NeuralPDE.var"#294#309"{ComponentArrays.ComponentVector{Float64, Vector{Float64}, Tuple{ComponentArrays.Axis{(depvar = ViewAxis(1:217, Axis(layer_1 = ViewAxis(1:64, Axis(weight = ViewAxis(1:56, ShapedAxis((8, 7), NamedTuple())), bias = ViewAxis(57:64, ShapedAxis((8, 1), NamedTuple())))), layer_2 = ViewAxis(65:136, Axis(weight = ViewAxis(1:64, ShapedAxis((8, 8), NamedTuple())), bias = ViewAxis(65:72, ShapedAxis((8, 1), NamedTuple())))), layer_3 = ViewAxis(137:208, Axis(weight = ViewAxis(1:64, ShapedAxis((8, 8), NamedTuple())), bias = ViewAxis(65:72, ShapedAxis((8, 1), NamedTuple())))), layer_4 = ViewAxis(209:217, Axis(weight = ViewAxis(1:8, ShapedAxis((1, 8), NamedTuple())), bias = ViewAxis(9:9, ShapedAxis((1, 1), NamedTuple())))))), p = 218:220)}}}}, args::Vector{NeuralPDE.var"#74#75"{NeuralPDE.var"#209#210"{RuntimeGeneratedFunctions.RuntimeGeneratedFunction{(:cord, Symbol("##θ#293"), :phi, :derivative, :integral, :u, :p), NeuralPDE.var"#_RGF_ModTag", NeuralPDE.var"#_RGF_ModTag", (0x7cf59a19, 0x0162d753, 0x260a99e5, 0xc4bc113e, 0x070b6155), Expr}, NeuralPDE.var"#12#13", NeuralPDE.var"#269#276"{NeuralPDE.var"#269#270#277"{typeof(NeuralPDE.numeric_derivative)}, Dict{Symbol, Int64}, Dict{Symbol, Int64}, GridTraining{Float64}}, typeof(NeuralPDE.numeric_derivative), NeuralPDE.Phi{Lux.Chain{NamedTuple{(:layer_1, :layer_2, :layer_3, :layer_4), Tuple{Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(identity), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}}}}, NamedTuple{(:layer_1, :layer_2, :layer_3, :layer_4), NTuple{4, NamedTuple{(), Tuple{}}}}}, Vector{Float64}}, Matrix{Float64}}})
    @ Zygote C:\Users\loren\.julia\packages\Zygote\YYT6v\src\lib\array.jl:183
 [32] _pullback
    @ C:\Users\loren\.julia\packages\Zygote\YYT6v\src\lib\array.jl:226 [inlined]
 [33] _pullback
    @ C:\Users\loren\.julia\packages\NeuralPDE\F4RlZ\src\discretize.jl:592 [inlined]
 [34] _pullback(::Zygote.Context{false}, ::NeuralPDE.var"#full_loss_function#308"{NeuralPDE.var"#null_nonadaptive_loss#115", Vector{NeuralPDE.var"#74#75"{NeuralPDE.var"#209#210"{RuntimeGeneratedFunctions.RuntimeGeneratedFunction{(:cord, Symbol("##θ#293"), :phi, :derivative, :integral, :u, :p), NeuralPDE.var"#_RGF_ModTag", NeuralPDE.var"#_RGF_ModTag", (0xad10bb64, 0x1d2657c2, 0xc9c67c1a, 0xbaf4020a, 0xc4a1861b), Expr}, NeuralPDE.var"#12#13", NeuralPDE.var"#269#276"{NeuralPDE.var"#269#270#277"{typeof(NeuralPDE.numeric_derivative)}, Dict{Symbol, Int64}, Dict{Symbol, Int64}, GridTraining{Float64}}, typeof(NeuralPDE.numeric_derivative), NeuralPDE.Phi{Lux.Chain{NamedTuple{(:layer_1, :layer_2, :layer_3, :layer_4), Tuple{Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(identity), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}}}}, NamedTuple{(:layer_1, :layer_2, :layer_3, :layer_4), NTuple{4, NamedTuple{(), Tuple{}}}}}, Vector{Float64}}, Matrix{Float64}}}, Vector{NeuralPDE.var"#74#75"{NeuralPDE.var"#209#210"{RuntimeGeneratedFunctions.RuntimeGeneratedFunction{(:cord, Symbol("##θ#293"), :phi, :derivative, :integral, :u, :p), NeuralPDE.var"#_RGF_ModTag", NeuralPDE.var"#_RGF_ModTag", (0x7cf59a19, 0x0162d753, 0x260a99e5, 0xc4bc113e, 0x070b6155), Expr}, NeuralPDE.var"#12#13", NeuralPDE.var"#269#276"{NeuralPDE.var"#269#270#277"{typeof(NeuralPDE.numeric_derivative)}, Dict{Symbol, Int64}, Dict{Symbol, Int64}, GridTraining{Float64}}, typeof(NeuralPDE.numeric_derivative), NeuralPDE.Phi{Lux.Chain{NamedTuple{(:layer_1, :layer_2, :layer_3, :layer_4), Tuple{Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(identity), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}}}}, NamedTuple{(:layer_1, :layer_2, :layer_3, :layer_4), NTuple{4, NamedTuple{(), Tuple{}}}}}, Vector{Float64}}, Matrix{Float64}}}, NeuralPDE.PINNRepresentation, Bool, Vector{Int64}, Int64, NeuralPDE.Phi{Lux.Chain{NamedTuple{(:layer_1, :layer_2, :layer_3, :layer_4), Tuple{Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(identity), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}}}}, NamedTuple{(:layer_1, :layer_2, :layer_3, :layer_4), NTuple{4, NamedTuple{(), Tuple{}}}}}, typeof(additional_loss), Bool, Vector{Float64}}, ::ComponentArrays.ComponentVector{Float64, Vector{Float64}, Tuple{ComponentArrays.Axis{(depvar = ViewAxis(1:217, Axis(layer_1 = ViewAxis(1:64, Axis(weight = ViewAxis(1:56, ShapedAxis((8, 7), NamedTuple())), bias = ViewAxis(57:64, ShapedAxis((8, 1), NamedTuple())))), layer_2 = ViewAxis(65:136, Axis(weight = ViewAxis(1:64, ShapedAxis((8, 8), NamedTuple())), bias = ViewAxis(65:72, ShapedAxis((8, 1), NamedTuple())))), layer_3 = ViewAxis(137:208, Axis(weight = ViewAxis(1:64, ShapedAxis((8, 8), NamedTuple())), bias = ViewAxis(65:72, ShapedAxis((8, 1), NamedTuple())))), layer_4 = ViewAxis(209:217, Axis(weight = ViewAxis(1:8, ShapedAxis((1, 8), NamedTuple())), bias = ViewAxis(9:9, ShapedAxis((1, 1), NamedTuple())))))), p = 218:220)}}}, ::SciMLBase.NullParameters)
    @ Zygote C:\Users\loren\.julia\packages\Zygote\YYT6v\src\compiler\interface2.jl:0
 [35] _apply(::Function, ::Vararg{Any})
    @ Core .\boot.jl:838
 [36] adjoint
    @ C:\Users\loren\.julia\packages\Zygote\YYT6v\src\lib\lib.jl:203 [inlined]
 [37] _pullback
    @ C:\Users\loren\.julia\packages\ZygoteRules\4nXuu\src\adjoint.jl:66 [inlined]
 [38] _pullback
    @ C:\Users\loren\.julia\packages\SciMLBase\QqtZA\src\scimlfunctions.jl:3580 [inlined]
 [39] _pullback(::Zygote.Context{false}, ::OptimizationFunction{true, Optimization.AutoZygote, NeuralPDE.var"#full_loss_function#308"{NeuralPDE.var"#null_nonadaptive_loss#115", Vector{NeuralPDE.var"#74#75"{NeuralPDE.var"#209#210"{RuntimeGeneratedFunctions.RuntimeGeneratedFunction{(:cord, Symbol("##θ#293"), :phi, :derivative, :integral, :u, :p), NeuralPDE.var"#_RGF_ModTag", NeuralPDE.var"#_RGF_ModTag", (0xad10bb64, 0x1d2657c2, 0xc9c67c1a, 0xbaf4020a, 0xc4a1861b), Expr}, NeuralPDE.var"#12#13", NeuralPDE.var"#269#276"{NeuralPDE.var"#269#270#277"{typeof(NeuralPDE.numeric_derivative)}, Dict{Symbol, Int64}, Dict{Symbol, Int64}, GridTraining{Float64}}, typeof(NeuralPDE.numeric_derivative), NeuralPDE.Phi{Lux.Chain{NamedTuple{(:layer_1, :layer_2, :layer_3, :layer_4), Tuple{Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(identity), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}}}}, NamedTuple{(:layer_1, :layer_2, :layer_3, :layer_4), NTuple{4, NamedTuple{(), Tuple{}}}}}, Vector{Float64}}, Matrix{Float64}}}, Vector{NeuralPDE.var"#74#75"{NeuralPDE.var"#209#210"{RuntimeGeneratedFunctions.RuntimeGeneratedFunction{(:cord, Symbol("##θ#293"), :phi, :derivative, :integral, :u, :p), NeuralPDE.var"#_RGF_ModTag", NeuralPDE.var"#_RGF_ModTag", (0x7cf59a19, 0x0162d753, 0x260a99e5, 0xc4bc113e, 0x070b6155), Expr}, NeuralPDE.var"#12#13", NeuralPDE.var"#269#276"{NeuralPDE.var"#269#270#277"{typeof(NeuralPDE.numeric_derivative)}, Dict{Symbol, Int64}, Dict{Symbol, Int64}, GridTraining{Float64}}, typeof(NeuralPDE.numeric_derivative), NeuralPDE.Phi{Lux.Chain{NamedTuple{(:layer_1, :layer_2, :layer_3, :layer_4), Tuple{Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(identity), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}}}}, NamedTuple{(:layer_1, :layer_2, :layer_3, :layer_4), NTuple{4, NamedTuple{(), Tuple{}}}}}, Vector{Float64}}, Matrix{Float64}}}, NeuralPDE.PINNRepresentation, Bool, Vector{Int64}, Int64, NeuralPDE.Phi{Lux.Chain{NamedTuple{(:layer_1, :layer_2, :layer_3, :layer_4), Tuple{Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(identity), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}}}}, NamedTuple{(:layer_1, :layer_2, :layer_3, :layer_4), NTuple{4, NamedTuple{(), Tuple{}}}}}, typeof(additional_loss), Bool, Vector{Float64}}, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, typeof(SciMLBase.DEFAULT_OBSERVED_NO_TIME), Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing}, ::ComponentArrays.ComponentVector{Float64, Vector{Float64}, Tuple{ComponentArrays.Axis{(depvar = ViewAxis(1:217, Axis(layer_1 = ViewAxis(1:64, Axis(weight = ViewAxis(1:56, ShapedAxis((8, 7), NamedTuple())), bias = ViewAxis(57:64, ShapedAxis((8, 1), NamedTuple())))), layer_2 = ViewAxis(65:136, Axis(weight = ViewAxis(1:64, ShapedAxis((8, 8), NamedTuple())), bias = ViewAxis(65:72, ShapedAxis((8, 1), NamedTuple())))), layer_3 = ViewAxis(137:208, Axis(weight = ViewAxis(1:64, ShapedAxis((8, 8), NamedTuple())), bias = ViewAxis(65:72, ShapedAxis((8, 1), NamedTuple())))), layer_4 = ViewAxis(209:217, Axis(weight = ViewAxis(1:8, ShapedAxis((1, 8), NamedTuple())), bias = ViewAxis(9:9, ShapedAxis((1, 1), NamedTuple())))))), p = 218:220)}}}, ::SciMLBase.NullParameters)
    @ Zygote C:\Users\loren\.julia\packages\Zygote\YYT6v\src\compiler\interface2.jl:0
 [40] _apply(::Function, ::Vararg{Any})
    @ Core .\boot.jl:838
 [41] adjoint
    @ C:\Users\loren\.julia\packages\Zygote\YYT6v\src\lib\lib.jl:203 [inlined]
 [42] _pullback
    @ C:\Users\loren\.julia\packages\ZygoteRules\4nXuu\src\adjoint.jl:66 [inlined]
 [43] _pullback
    @ C:\Users\loren\.julia\packages\Optimization\vFala\src\function\zygote.jl:72 [inlined]
 [44] _pullback(ctx::Zygote.Context{false}, f::Optimization.var"#227#236"{OptimizationFunction{true, Optimization.AutoZygote, NeuralPDE.var"#full_loss_function#308"{NeuralPDE.var"#null_nonadaptive_loss#115", Vector{NeuralPDE.var"#74#75"{NeuralPDE.var"#209#210"{RuntimeGeneratedFunctions.RuntimeGeneratedFunction{(:cord, Symbol("##θ#293"), :phi, :derivative, :integral, :u, :p), NeuralPDE.var"#_RGF_ModTag", NeuralPDE.var"#_RGF_ModTag", (0xad10bb64, 0x1d2657c2, 0xc9c67c1a, 0xbaf4020a, 0xc4a1861b), Expr}, NeuralPDE.var"#12#13", NeuralPDE.var"#269#276"{NeuralPDE.var"#269#270#277"{typeof(NeuralPDE.numeric_derivative)}, Dict{Symbol, Int64}, Dict{Symbol, Int64}, GridTraining{Float64}}, typeof(NeuralPDE.numeric_derivative), NeuralPDE.Phi{Lux.Chain{NamedTuple{(:layer_1, :layer_2, :layer_3, :layer_4), Tuple{Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(identity), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}}}}, NamedTuple{(:layer_1, :layer_2, :layer_3, :layer_4), NTuple{4, NamedTuple{(), Tuple{}}}}}, Vector{Float64}}, Matrix{Float64}}}, Vector{NeuralPDE.var"#74#75"{NeuralPDE.var"#209#210"{RuntimeGeneratedFunctions.RuntimeGeneratedFunction{(:cord, Symbol("##θ#293"), :phi, :derivative, :integral, :u, :p), NeuralPDE.var"#_RGF_ModTag", NeuralPDE.var"#_RGF_ModTag", (0x7cf59a19, 0x0162d753, 0x260a99e5, 0xc4bc113e, 0x070b6155), Expr}, NeuralPDE.var"#12#13", NeuralPDE.var"#269#276"{NeuralPDE.var"#269#270#277"{typeof(NeuralPDE.numeric_derivative)}, Dict{Symbol, Int64}, Dict{Symbol, Int64}, GridTraining{Float64}}, typeof(NeuralPDE.numeric_derivative), NeuralPDE.Phi{Lux.Chain{NamedTuple{(:layer_1, :layer_2, :layer_3, :layer_4), Tuple{Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(identity), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}}}}, NamedTuple{(:layer_1, :layer_2, :layer_3, :layer_4), NTuple{4, NamedTuple{(), Tuple{}}}}}, Vector{Float64}}, Matrix{Float64}}}, NeuralPDE.PINNRepresentation, Bool, Vector{Int64}, Int64, NeuralPDE.Phi{Lux.Chain{NamedTuple{(:layer_1, :layer_2, :layer_3, :layer_4), Tuple{Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(identity), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}}}}, NamedTuple{(:layer_1, :layer_2, :layer_3, :layer_4), NTuple{4, NamedTuple{(), Tuple{}}}}}, typeof(additional_loss), Bool, Vector{Float64}}, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, typeof(SciMLBase.DEFAULT_OBSERVED_NO_TIME), Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing}, Optimization.ReInitCache{ComponentArrays.ComponentVector{Float64, Vector{Float64}, Tuple{ComponentArrays.Axis{(depvar = ViewAxis(1:217, Axis(layer_1 = ViewAxis(1:64, Axis(weight = ViewAxis(1:56, ShapedAxis((8, 7), NamedTuple())), bias = ViewAxis(57:64, ShapedAxis((8, 1), NamedTuple())))), layer_2 = ViewAxis(65:136, Axis(weight = ViewAxis(1:64, ShapedAxis((8, 8), NamedTuple())), bias = ViewAxis(65:72, ShapedAxis((8, 1), NamedTuple())))), layer_3 = ViewAxis(137:208, Axis(weight = ViewAxis(1:64, ShapedAxis((8, 8), NamedTuple())), bias = ViewAxis(65:72, ShapedAxis((8, 1), NamedTuple())))), layer_4 = ViewAxis(209:217, Axis(weight = ViewAxis(1:8, ShapedAxis((1, 8), NamedTuple())), bias = ViewAxis(9:9, ShapedAxis((1, 1), NamedTuple())))))), p = 218:220)}}}, SciMLBase.NullParameters}}, args::ComponentArrays.ComponentVector{Float64, Vector{Float64}, Tuple{ComponentArrays.Axis{(depvar = ViewAxis(1:217, Axis(layer_1 = ViewAxis(1:64, Axis(weight = ViewAxis(1:56, ShapedAxis((8, 7), NamedTuple())), bias = ViewAxis(57:64, ShapedAxis((8, 1), NamedTuple())))), layer_2 = ViewAxis(65:136, Axis(weight = ViewAxis(1:64, ShapedAxis((8, 8), NamedTuple())), bias = ViewAxis(65:72, ShapedAxis((8, 1), NamedTuple())))), layer_3 = ViewAxis(137:208, Axis(weight = ViewAxis(1:64, ShapedAxis((8, 8), NamedTuple())), bias = ViewAxis(65:72, ShapedAxis((8, 1), NamedTuple())))), layer_4 = ViewAxis(209:217, Axis(weight = ViewAxis(1:8, ShapedAxis((1, 8), NamedTuple())), bias = ViewAxis(9:9, ShapedAxis((1, 1), NamedTuple())))))), p = 218:220)}}})
    @ Zygote C:\Users\loren\.julia\packages\Zygote\YYT6v\src\compiler\interface2.jl:0
 [45] _apply(::Function, ::Vararg{Any})
    @ Core .\boot.jl:838
 [46] adjoint
    @ C:\Users\loren\.julia\packages\Zygote\YYT6v\src\lib\lib.jl:203 [inlined]
 [47] _pullback
    @ C:\Users\loren\.julia\packages\ZygoteRules\4nXuu\src\adjoint.jl:66 [inlined]
 [48] _pullback
    @ C:\Users\loren\.julia\packages\Optimization\vFala\src\function\zygote.jl:76 [inlined]
 [49] _pullback(ctx::Zygote.Context{false}, f::Optimization.var"#229#238"{Tuple{}, Optimization.var"#227#236"{OptimizationFunction{true, Optimization.AutoZygote, NeuralPDE.var"#full_loss_function#308"{NeuralPDE.var"#null_nonadaptive_loss#115", Vector{NeuralPDE.var"#74#75"{NeuralPDE.var"#209#210"{RuntimeGeneratedFunctions.RuntimeGeneratedFunction{(:cord, Symbol("##θ#293"), :phi, :derivative, :integral, :u, :p), NeuralPDE.var"#_RGF_ModTag", NeuralPDE.var"#_RGF_ModTag", (0xad10bb64, 0x1d2657c2, 0xc9c67c1a, 0xbaf4020a, 0xc4a1861b), Expr}, NeuralPDE.var"#12#13", NeuralPDE.var"#269#276"{NeuralPDE.var"#269#270#277"{typeof(NeuralPDE.numeric_derivative)}, Dict{Symbol, Int64}, Dict{Symbol, Int64}, GridTraining{Float64}}, typeof(NeuralPDE.numeric_derivative), NeuralPDE.Phi{Lux.Chain{NamedTuple{(:layer_1, :layer_2, :layer_3, :layer_4), Tuple{Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(identity), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}}}}, NamedTuple{(:layer_1, :layer_2, :layer_3, :layer_4), NTuple{4, NamedTuple{(), Tuple{}}}}}, Vector{Float64}}, Matrix{Float64}}}, Vector{NeuralPDE.var"#74#75"{NeuralPDE.var"#209#210"{RuntimeGeneratedFunctions.RuntimeGeneratedFunction{(:cord, Symbol("##θ#293"), :phi, :derivative, :integral, :u, :p), NeuralPDE.var"#_RGF_ModTag", NeuralPDE.var"#_RGF_ModTag", (0x7cf59a19, 0x0162d753, 0x260a99e5, 0xc4bc113e, 0x070b6155), Expr}, NeuralPDE.var"#12#13", NeuralPDE.var"#269#276"{NeuralPDE.var"#269#270#277"{typeof(NeuralPDE.numeric_derivative)}, Dict{Symbol, Int64}, Dict{Symbol, Int64}, GridTraining{Float64}}, typeof(NeuralPDE.numeric_derivative), NeuralPDE.Phi{Lux.Chain{NamedTuple{(:layer_1, :layer_2, :layer_3, :layer_4), Tuple{Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(identity), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}}}}, NamedTuple{(:layer_1, :layer_2, :layer_3, :layer_4), NTuple{4, NamedTuple{(), Tuple{}}}}}, Vector{Float64}}, Matrix{Float64}}}, NeuralPDE.PINNRepresentation, Bool, Vector{Int64}, Int64, NeuralPDE.Phi{Lux.Chain{NamedTuple{(:layer_1, :layer_2, :layer_3, :layer_4), Tuple{Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(identity), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}}}}, NamedTuple{(:layer_1, :layer_2, :layer_3, :layer_4), NTuple{4, NamedTuple{(), Tuple{}}}}}, typeof(additional_loss), Bool, Vector{Float64}}, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, typeof(SciMLBase.DEFAULT_OBSERVED_NO_TIME), Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing}, Optimization.ReInitCache{ComponentArrays.ComponentVector{Float64, Vector{Float64}, Tuple{ComponentArrays.Axis{(depvar = ViewAxis(1:217, Axis(layer_1 = ViewAxis(1:64, Axis(weight = ViewAxis(1:56, ShapedAxis((8, 7), NamedTuple())), bias = ViewAxis(57:64, ShapedAxis((8, 1), NamedTuple())))), layer_2 = ViewAxis(65:136, Axis(weight = ViewAxis(1:64, ShapedAxis((8, 8), NamedTuple())), bias = ViewAxis(65:72, ShapedAxis((8, 1), NamedTuple())))), layer_3 = ViewAxis(137:208, Axis(weight = ViewAxis(1:64, ShapedAxis((8, 8), NamedTuple())), bias = ViewAxis(65:72, ShapedAxis((8, 1), NamedTuple())))), layer_4 = ViewAxis(209:217, Axis(weight = ViewAxis(1:8, ShapedAxis((1, 8), NamedTuple())), bias = ViewAxis(9:9, ShapedAxis((1, 1), NamedTuple())))))), p = 218:220)}}}, SciMLBase.NullParameters}}}, args::ComponentArrays.ComponentVector{Float64, Vector{Float64}, Tuple{ComponentArrays.Axis{(depvar = ViewAxis(1:217, Axis(layer_1 = ViewAxis(1:64, Axis(weight = ViewAxis(1:56, ShapedAxis((8, 7), NamedTuple())), bias = ViewAxis(57:64, ShapedAxis((8, 1), NamedTuple())))), layer_2 = ViewAxis(65:136, Axis(weight = ViewAxis(1:64, ShapedAxis((8, 8), NamedTuple())), bias = ViewAxis(65:72, ShapedAxis((8, 1), NamedTuple())))), layer_3 = ViewAxis(137:208, Axis(weight = ViewAxis(1:64, ShapedAxis((8, 8), NamedTuple())), bias = ViewAxis(65:72, ShapedAxis((8, 1), NamedTuple())))), layer_4 = ViewAxis(209:217, Axis(weight = ViewAxis(1:8, ShapedAxis((1, 8), NamedTuple())), bias = ViewAxis(9:9, ShapedAxis((1, 1), NamedTuple())))))), p = 218:220)}}})
    @ Zygote C:\Users\loren\.julia\packages\Zygote\YYT6v\src\compiler\interface2.jl:0
 [50] pullback(f::Function, cx::Zygote.Context{false}, args::ComponentArrays.ComponentVector{Float64, Vector{Float64}, Tuple{ComponentArrays.Axis{(depvar = ViewAxis(1:217, Axis(layer_1 = ViewAxis(1:64, Axis(weight = ViewAxis(1:56, ShapedAxis((8, 7), NamedTuple())), bias = ViewAxis(57:64, ShapedAxis((8, 1), NamedTuple())))), layer_2 = ViewAxis(65:136, Axis(weight = ViewAxis(1:64, ShapedAxis((8, 8), NamedTuple())), bias = ViewAxis(65:72, ShapedAxis((8, 1), NamedTuple())))), layer_3 = ViewAxis(137:208, Axis(weight = ViewAxis(1:64, ShapedAxis((8, 8), NamedTuple())), bias = ViewAxis(65:72, ShapedAxis((8, 1), NamedTuple())))), layer_4 = ViewAxis(209:217, Axis(weight = ViewAxis(1:8, ShapedAxis((1, 8), NamedTuple())), bias = ViewAxis(9:9, ShapedAxis((1, 1), NamedTuple())))))), p = 218:220)}}})
    @ Zygote C:\Users\loren\.julia\packages\Zygote\YYT6v\src\compiler\interface.jl:44
 [51] pullback
    @ C:\Users\loren\.julia\packages\Zygote\YYT6v\src\compiler\interface.jl:42 [inlined]
 [52] gradient(f::Function, args::ComponentArrays.ComponentVector{Float64, Vector{Float64}, Tuple{ComponentArrays.Axis{(depvar = ViewAxis(1:217, Axis(layer_1 = ViewAxis(1:64, Axis(weight = ViewAxis(1:56, ShapedAxis((8, 7), NamedTuple())), bias = ViewAxis(57:64, ShapedAxis((8, 1), NamedTuple())))), layer_2 = ViewAxis(65:136, Axis(weight = ViewAxis(1:64, ShapedAxis((8, 8), NamedTuple())), bias = ViewAxis(65:72, ShapedAxis((8, 1), NamedTuple())))), layer_3 = ViewAxis(137:208, Axis(weight = ViewAxis(1:64, ShapedAxis((8, 8), NamedTuple())), bias = ViewAxis(65:72, ShapedAxis((8, 1), NamedTuple())))), layer_4 = ViewAxis(209:217, Axis(weight = ViewAxis(1:8, ShapedAxis((1, 8), NamedTuple())), bias = ViewAxis(9:9, ShapedAxis((1, 1), NamedTuple())))))), p = 218:220)}}})
    @ Zygote C:\Users\loren\.julia\packages\Zygote\YYT6v\src\compiler\interface.jl:96
 [53] (::Optimization.var"#228#237"{Optimization.var"#227#236"{OptimizationFunction{true, Optimization.AutoZygote, NeuralPDE.var"#full_loss_function#308"{NeuralPDE.var"#null_nonadaptive_loss#115", Vector{NeuralPDE.var"#74#75"{NeuralPDE.var"#209#210"{RuntimeGeneratedFunctions.RuntimeGeneratedFunction{(:cord, Symbol("##θ#293"), :phi, :derivative, :integral, :u, :p), NeuralPDE.var"#_RGF_ModTag", NeuralPDE.var"#_RGF_ModTag", (0xad10bb64, 0x1d2657c2, 0xc9c67c1a, 0xbaf4020a, 0xc4a1861b), Expr}, NeuralPDE.var"#12#13", NeuralPDE.var"#269#276"{NeuralPDE.var"#269#270#277"{typeof(NeuralPDE.numeric_derivative)}, Dict{Symbol, Int64}, Dict{Symbol, Int64}, GridTraining{Float64}}, typeof(NeuralPDE.numeric_derivative), NeuralPDE.Phi{Lux.Chain{NamedTuple{(:layer_1, :layer_2, :layer_3, :layer_4), Tuple{Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(identity), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}}}}, NamedTuple{(:layer_1, :layer_2, :layer_3, :layer_4), NTuple{4, NamedTuple{(), Tuple{}}}}}, Vector{Float64}}, Matrix{Float64}}}, Vector{NeuralPDE.var"#74#75"{NeuralPDE.var"#209#210"{RuntimeGeneratedFunctions.RuntimeGeneratedFunction{(:cord, Symbol("##θ#293"), :phi, :derivative, :integral, :u, :p), NeuralPDE.var"#_RGF_ModTag", NeuralPDE.var"#_RGF_ModTag", (0x7cf59a19, 0x0162d753, 0x260a99e5, 0xc4bc113e, 0x070b6155), Expr}, NeuralPDE.var"#12#13", NeuralPDE.var"#269#276"{NeuralPDE.var"#269#270#277"{typeof(NeuralPDE.numeric_derivative)}, Dict{Symbol, Int64}, Dict{Symbol, Int64}, GridTraining{Float64}}, typeof(NeuralPDE.numeric_derivative), NeuralPDE.Phi{Lux.Chain{NamedTuple{(:layer_1, :layer_2, :layer_3, :layer_4), Tuple{Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(identity), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}}}}, NamedTuple{(:layer_1, :layer_2, :layer_3, :layer_4), NTuple{4, NamedTuple{(), Tuple{}}}}}, Vector{Float64}}, Matrix{Float64}}}, NeuralPDE.PINNRepresentation, Bool, Vector{Int64}, Int64, NeuralPDE.Phi{Lux.Chain{NamedTuple{(:layer_1, :layer_2, :layer_3, :layer_4), Tuple{Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(identity), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}}}}, NamedTuple{(:layer_1, :layer_2, :layer_3, :layer_4), NTuple{4, NamedTuple{(), Tuple{}}}}}, typeof(additional_loss), Bool, Vector{Float64}}, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, typeof(SciMLBase.DEFAULT_OBSERVED_NO_TIME), Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing}, Optimization.ReInitCache{ComponentArrays.ComponentVector{Float64, Vector{Float64}, Tuple{ComponentArrays.Axis{(depvar = ViewAxis(1:217, Axis(layer_1 = ViewAxis(1:64, Axis(weight = ViewAxis(1:56, ShapedAxis((8, 7), NamedTuple())), bias = ViewAxis(57:64, ShapedAxis((8, 1), NamedTuple())))), layer_2 = ViewAxis(65:136, Axis(weight = ViewAxis(1:64, ShapedAxis((8, 8), NamedTuple())), bias = ViewAxis(65:72, ShapedAxis((8, 1), NamedTuple())))), layer_3 = ViewAxis(137:208, Axis(weight = ViewAxis(1:64, ShapedAxis((8, 8), NamedTuple())), bias = ViewAxis(65:72, ShapedAxis((8, 1), NamedTuple())))), layer_4 = ViewAxis(209:217, Axis(weight = ViewAxis(1:8, ShapedAxis((1, 8), NamedTuple())), bias = ViewAxis(9:9, ShapedAxis((1, 1), NamedTuple())))))), p = 218:220)}}}, SciMLBase.NullParameters}}})(::ComponentArrays.ComponentVector{Float64, Vector{Float64}, Tuple{ComponentArrays.Axis{(depvar = ViewAxis(1:217, Axis(layer_1 = ViewAxis(1:64, Axis(weight = ViewAxis(1:56, ShapedAxis((8, 7), NamedTuple())), bias = ViewAxis(57:64, ShapedAxis((8, 1), NamedTuple())))), layer_2 = ViewAxis(65:136, Axis(weight = ViewAxis(1:64, ShapedAxis((8, 8), NamedTuple())), bias = ViewAxis(65:72, ShapedAxis((8, 1), NamedTuple())))), layer_3 = ViewAxis(137:208, Axis(weight = ViewAxis(1:64, ShapedAxis((8, 8), NamedTuple())), bias = ViewAxis(65:72, ShapedAxis((8, 1), NamedTuple())))), layer_4 = ViewAxis(209:217, Axis(weight = ViewAxis(1:8, ShapedAxis((1, 8), NamedTuple())), bias = ViewAxis(9:9, ShapedAxis((1, 1), NamedTuple())))))), p = 218:220)}}}, ::ComponentArrays.ComponentVector{Float64, Vector{Float64}, Tuple{ComponentArrays.Axis{(depvar = ViewAxis(1:217, Axis(layer_1 = ViewAxis(1:64, Axis(weight = ViewAxis(1:56, ShapedAxis((8, 7), NamedTuple())), bias = ViewAxis(57:64, ShapedAxis((8, 1), NamedTuple())))), layer_2 = ViewAxis(65:136, Axis(weight = ViewAxis(1:64, ShapedAxis((8, 8), NamedTuple())), bias = ViewAxis(65:72, ShapedAxis((8, 1), NamedTuple())))), layer_3 = ViewAxis(137:208, Axis(weight = ViewAxis(1:64, ShapedAxis((8, 8), NamedTuple())), bias = ViewAxis(65:72, ShapedAxis((8, 1), NamedTuple())))), layer_4 = ViewAxis(209:217, Axis(weight = ViewAxis(1:8, ShapedAxis((1, 8), NamedTuple())), bias = ViewAxis(9:9, ShapedAxis((1, 1), NamedTuple())))))), p = 218:220)}}})
    @ Optimization C:\Users\loren\.julia\packages\Optimization\vFala\src\function\zygote.jl:74
 [54] (::OptimizationOptimJL.var"#10#16"{OptimizationOptimJL.OptimJLOptimizationCache{OptimizationFunction{false, Optimization.AutoZygote, OptimizationFunction{true, Optimization.AutoZygote, NeuralPDE.var"#full_loss_function#308"{NeuralPDE.var"#null_nonadaptive_loss#115", Vector{NeuralPDE.var"#74#75"{NeuralPDE.var"#209#210"{RuntimeGeneratedFunctions.RuntimeGeneratedFunction{(:cord, Symbol("##θ#293"), :phi, :derivative, :integral, :u, :p), NeuralPDE.var"#_RGF_ModTag", NeuralPDE.var"#_RGF_ModTag", (0xad10bb64, 0x1d2657c2, 0xc9c67c1a, 0xbaf4020a, 0xc4a1861b), Expr}, NeuralPDE.var"#12#13", NeuralPDE.var"#269#276"{NeuralPDE.var"#269#270#277"{typeof(NeuralPDE.numeric_derivative)}, Dict{Symbol, Int64}, Dict{Symbol, Int64}, GridTraining{Float64}}, typeof(NeuralPDE.numeric_derivative), NeuralPDE.Phi{Lux.Chain{NamedTuple{(:layer_1, :layer_2, :layer_3, :layer_4), Tuple{Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(identity), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}}}}, NamedTuple{(:layer_1, :layer_2, :layer_3, :layer_4), NTuple{4, NamedTuple{(), Tuple{}}}}}, Vector{Float64}}, Matrix{Float64}}}, Vector{NeuralPDE.var"#74#75"{NeuralPDE.var"#209#210"{RuntimeGeneratedFunctions.RuntimeGeneratedFunction{(:cord, Symbol("##θ#293"), :phi, :derivative, :integral, :u, :p), NeuralPDE.var"#_RGF_ModTag", NeuralPDE.var"#_RGF_ModTag", (0x7cf59a19, 0x0162d753, 0x260a99e5, 0xc4bc113e, 0x070b6155), Expr}, NeuralPDE.var"#12#13", NeuralPDE.var"#269#276"{NeuralPDE.var"#269#270#277"{typeof(NeuralPDE.numeric_derivative)}, Dict{Symbol, Int64}, Dict{Symbol, Int64}, GridTraining{Float64}}, typeof(NeuralPDE.numeric_derivative), NeuralPDE.Phi{Lux.Chain{NamedTuple{(:layer_1, :layer_2, :layer_3, :layer_4), Tuple{Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(identity), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}}}}, NamedTuple{(:layer_1, :layer_2, :layer_3, :layer_4), NTuple{4, NamedTuple{(), Tuple{}}}}}, Vector{Float64}}, Matrix{Float64}}}, NeuralPDE.PINNRepresentation, Bool, Vector{Int64}, Int64, NeuralPDE.Phi{Lux.Chain{NamedTuple{(:layer_1, :layer_2, :layer_3, :layer_4), Tuple{Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(identity), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}}}}, NamedTuple{(:layer_1, :layer_2, :layer_3, :layer_4), NTuple{4, NamedTuple{(), Tuple{}}}}}, typeof(additional_loss), Bool, Vector{Float64}}, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, typeof(SciMLBase.DEFAULT_OBSERVED_NO_TIME), Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing}, Optimization.var"#228#237"{Optimization.var"#227#236"{OptimizationFunction{true, Optimization.AutoZygote, NeuralPDE.var"#full_loss_function#308"{NeuralPDE.var"#null_nonadaptive_loss#115", Vector{NeuralPDE.var"#74#75"{NeuralPDE.var"#209#210"{RuntimeGeneratedFunctions.RuntimeGeneratedFunction{(:cord, Symbol("##θ#293"), :phi, :derivative, :integral, :u, :p), NeuralPDE.var"#_RGF_ModTag", NeuralPDE.var"#_RGF_ModTag", (0xad10bb64, 0x1d2657c2, 0xc9c67c1a, 0xbaf4020a, 0xc4a1861b), Expr}, NeuralPDE.var"#12#13", NeuralPDE.var"#269#276"{NeuralPDE.var"#269#270#277"{typeof(NeuralPDE.numeric_derivative)}, Dict{Symbol, Int64}, Dict{Symbol, Int64}, GridTraining{Float64}}, typeof(NeuralPDE.numeric_derivative), NeuralPDE.Phi{Lux.Chain{NamedTuple{(:layer_1, :layer_2, :layer_3, :layer_4), Tuple{Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(identity), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}}}}, NamedTuple{(:layer_1, :layer_2, :layer_3, :layer_4), NTuple{4, NamedTuple{(), Tuple{}}}}}, Vector{Float64}}, Matrix{Float64}}}, Vector{NeuralPDE.var"#74#75"{NeuralPDE.var"#209#210"{RuntimeGeneratedFunctions.RuntimeGeneratedFunction{(:cord, Symbol("##θ#293"), :phi, :derivative, :integral, :u, :p), NeuralPDE.var"#_RGF_ModTag", NeuralPDE.var"#_RGF_ModTag", (0x7cf59a19, 0x0162d753, 0x260a99e5, 0xc4bc113e, 0x070b6155), Expr}, NeuralPDE.var"#12#13", NeuralPDE.var"#269#276"{NeuralPDE.var"#269#270#277"{typeof(NeuralPDE.numeric_derivative)}, Dict{Symbol, Int64}, Dict{Symbol, Int64}, GridTraining{Float64}}, typeof(NeuralPDE.numeric_derivative), NeuralPDE.Phi{Lux.Chain{NamedTuple{(:layer_1, :layer_2, :layer_3, :layer_4), Tuple{Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(identity), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}}}}, NamedTuple{(:layer_1, :layer_2, :layer_3, :layer_4), NTuple{4, NamedTuple{(), Tuple{}}}}}, Vector{Float64}}, Matrix{Float64}}}, NeuralPDE.PINNRepresentation, Bool, Vector{Int64}, Int64, NeuralPDE.Phi{Lux.Chain{NamedTuple{(:layer_1, :layer_2, :layer_3, :layer_4), Tuple{Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(identity), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}}}}, NamedTuple{(:layer_1, :layer_2, :layer_3, :layer_4), NTuple{4, NamedTuple{(), Tuple{}}}}}, typeof(additional_loss), Bool, Vector{Float64}}, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, typeof(SciMLBase.DEFAULT_OBSERVED_NO_TIME), Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing}, Optimization.ReInitCache{ComponentArrays.ComponentVector{Float64, Vector{Float64}, Tuple{ComponentArrays.Axis{(depvar = ViewAxis(1:217, Axis(layer_1 = ViewAxis(1:64, Axis(weight = ViewAxis(1:56, ShapedAxis((8, 7), NamedTuple())), bias = ViewAxis(57:64, ShapedAxis((8, 1), NamedTuple())))), layer_2 = ViewAxis(65:136, Axis(weight = ViewAxis(1:64, ShapedAxis((8, 8), NamedTuple())), bias = ViewAxis(65:72, ShapedAxis((8, 1), NamedTuple())))), layer_3 = ViewAxis(137:208, Axis(weight = ViewAxis(1:64, ShapedAxis((8, 8), NamedTuple())), bias = ViewAxis(65:72, ShapedAxis((8, 1), NamedTuple())))), layer_4 = ViewAxis(209:217, Axis(weight = ViewAxis(1:8, ShapedAxis((1, 8), NamedTuple())), bias = ViewAxis(9:9, ShapedAxis((1, 1), NamedTuple())))))), p = 218:220)}}}, SciMLBase.NullParameters}}}, Optimization.var"#231#240"{Optimization.var"#227#236"{OptimizationFunction{true, Optimization.AutoZygote, NeuralPDE.var"#full_loss_function#308"{NeuralPDE.var"#null_nonadaptive_loss#115", Vector{NeuralPDE.var"#74#75"{NeuralPDE.var"#209#210"{RuntimeGeneratedFunctions.RuntimeGeneratedFunction{(:cord, Symbol("##θ#293"), :phi, :derivative, :integral, :u, :p), NeuralPDE.var"#_RGF_ModTag", NeuralPDE.var"#_RGF_ModTag", (0xad10bb64, 0x1d2657c2, 0xc9c67c1a, 0xbaf4020a, 0xc4a1861b), Expr}, NeuralPDE.var"#12#13", NeuralPDE.var"#269#276"{NeuralPDE.var"#269#270#277"{typeof(NeuralPDE.numeric_derivative)}, Dict{Symbol, Int64}, Dict{Symbol, Int64}, GridTraining{Float64}}, typeof(NeuralPDE.numeric_derivative), NeuralPDE.Phi{Lux.Chain{NamedTuple{(:layer_1, :layer_2, :layer_3, :layer_4), Tuple{Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(identity), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}}}}, NamedTuple{(:layer_1, :layer_2, :layer_3, :layer_4), NTuple{4, NamedTuple{(), Tuple{}}}}}, Vector{Float64}}, Matrix{Float64}}}, Vector{NeuralPDE.var"#74#75"{NeuralPDE.var"#209#210"{RuntimeGeneratedFunctions.RuntimeGeneratedFunction{(:cord, Symbol("##θ#293"), :phi, :derivative, :integral, :u, :p), NeuralPDE.var"#_RGF_ModTag", NeuralPDE.var"#_RGF_ModTag", (0x7cf59a19, 0x0162d753, 0x260a99e5, 0xc4bc113e, 0x070b6155), Expr}, NeuralPDE.var"#12#13", NeuralPDE.var"#269#276"{NeuralPDE.var"#269#270#277"{typeof(NeuralPDE.numeric_derivative)}, Dict{Symbol, Int64}, Dict{Symbol, Int64}, GridTraining{Float64}}, typeof(NeuralPDE.numeric_derivative), NeuralPDE.Phi{Lux.Chain{NamedTuple{(:layer_1, :layer_2, :layer_3, :layer_4), Tuple{Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(identity), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}}}}, NamedTuple{(:layer_1, :layer_2, :layer_3, :layer_4), NTuple{4, NamedTuple{(), Tuple{}}}}}, Vector{Float64}}, Matrix{Float64}}}, NeuralPDE.PINNRepresentation, Bool, Vector{Int64}, Int64, NeuralPDE.Phi{Lux.Chain{NamedTuple{(:layer_1, :layer_2, :layer_3, :layer_4), Tuple{Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(identity), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}}}}, NamedTuple{(:layer_1, :layer_2, :layer_3, :layer_4), NTuple{4, NamedTuple{(), Tuple{}}}}}, typeof(additional_loss), Bool, Vector{Float64}}, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, typeof(SciMLBase.DEFAULT_OBSERVED_NO_TIME), Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing}, Optimization.ReInitCache{ComponentArrays.ComponentVector{Float64, Vector{Float64}, Tuple{ComponentArrays.Axis{(depvar = ViewAxis(1:217, Axis(layer_1 = ViewAxis(1:64, Axis(weight = ViewAxis(1:56, ShapedAxis((8, 7), NamedTuple())), bias = ViewAxis(57:64, ShapedAxis((8, 1), NamedTuple())))), layer_2 = ViewAxis(65:136, Axis(weight = ViewAxis(1:64, ShapedAxis((8, 8), NamedTuple())), bias = ViewAxis(65:72, ShapedAxis((8, 1), NamedTuple())))), layer_3 = ViewAxis(137:208, Axis(weight = ViewAxis(1:64, ShapedAxis((8, 8), NamedTuple())), bias = ViewAxis(65:72, ShapedAxis((8, 1), NamedTuple())))), layer_4 = ViewAxis(209:217, Axis(weight = ViewAxis(1:8, ShapedAxis((1, 8), NamedTuple())), bias = ViewAxis(9:9, ShapedAxis((1, 1), NamedTuple())))))), p = 218:220)}}}, SciMLBase.NullParameters}}}, Optimization.var"#235#244", Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, typeof(SciMLBase.DEFAULT_OBSERVED_NO_TIME), Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing}, Optimization.ReInitCache{ComponentArrays.ComponentVector{Float64, Vector{Float64}, Tuple{ComponentArrays.Axis{(depvar = ViewAxis(1:217, Axis(layer_1 = ViewAxis(1:64, Axis(weight = ViewAxis(1:56, ShapedAxis((8, 7), NamedTuple())), bias = ViewAxis(57:64, ShapedAxis((8, 1), NamedTuple())))), layer_2 = ViewAxis(65:136, Axis(weight = ViewAxis(1:64, ShapedAxis((8, 8), NamedTuple())), bias = ViewAxis(65:72, ShapedAxis((8, 1), NamedTuple())))), layer_3 = ViewAxis(137:208, Axis(weight = ViewAxis(1:64, ShapedAxis((8, 8), NamedTuple())), bias = ViewAxis(65:72, ShapedAxis((8, 1), NamedTuple())))), layer_4 = ViewAxis(209:217, Axis(weight = ViewAxis(1:8, ShapedAxis((1, 8), NamedTuple())), bias = ViewAxis(9:9, ShapedAxis((1, 1), NamedTuple())))))), p = 218:220)}}}, SciMLBase.NullParameters}, Nothing, Nothing, Nothing, Nothing, Nothing, BFGS{LineSearches.InitialStatic{Float64}, LineSearches.HagerZhang{Float64, Base.RefValue{Bool}}, Nothing, Nothing, Flat}, Base.Iterators.Cycle{Tuple{Optimization.NullData}}, Bool, var"#3#4"}, OptimizationOptimJL.var"#9#15"{OptimizationOptimJL.OptimJLOptimizationCache{OptimizationFunction{false, Optimization.AutoZygote, OptimizationFunction{true, Optimization.AutoZygote, NeuralPDE.var"#full_loss_function#308"{NeuralPDE.var"#null_nonadaptive_loss#115", Vector{NeuralPDE.var"#74#75"{NeuralPDE.var"#209#210"{RuntimeGeneratedFunctions.RuntimeGeneratedFunction{(:cord, Symbol("##θ#293"), :phi, :derivative, :integral, :u, :p), NeuralPDE.var"#_RGF_ModTag", NeuralPDE.var"#_RGF_ModTag", (0xad10bb64, 0x1d2657c2, 0xc9c67c1a, 0xbaf4020a, 0xc4a1861b), Expr}, NeuralPDE.var"#12#13", NeuralPDE.var"#269#276"{NeuralPDE.var"#269#270#277"{typeof(NeuralPDE.numeric_derivative)}, Dict{Symbol, Int64}, Dict{Symbol, Int64}, GridTraining{Float64}}, typeof(NeuralPDE.numeric_derivative), NeuralPDE.Phi{Lux.Chain{NamedTuple{(:layer_1, :layer_2, :layer_3, :layer_4), Tuple{Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(identity), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}}}}, NamedTuple{(:layer_1, :layer_2, :layer_3, :layer_4), NTuple{4, NamedTuple{(), Tuple{}}}}}, Vector{Float64}}, Matrix{Float64}}}, Vector{NeuralPDE.var"#74#75"{NeuralPDE.var"#209#210"{RuntimeGeneratedFunctions.RuntimeGeneratedFunction{(:cord, Symbol("##θ#293"), :phi, :derivative, :integral, :u, :p), NeuralPDE.var"#_RGF_ModTag", NeuralPDE.var"#_RGF_ModTag", (0x7cf59a19, 0x0162d753, 0x260a99e5, 0xc4bc113e, 0x070b6155), Expr}, NeuralPDE.var"#12#13", NeuralPDE.var"#269#276"{NeuralPDE.var"#269#270#277"{typeof(NeuralPDE.numeric_derivative)}, Dict{Symbol, Int64}, Dict{Symbol, Int64}, GridTraining{Float64}}, typeof(NeuralPDE.numeric_derivative), NeuralPDE.Phi{Lux.Chain{NamedTuple{(:layer_1, :layer_2, :layer_3, :layer_4), Tuple{Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(identity), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}}}}, NamedTuple{(:layer_1, :layer_2, :layer_3, :layer_4), NTuple{4, NamedTuple{(), Tuple{}}}}}, Vector{Float64}}, Matrix{Float64}}}, NeuralPDE.PINNRepresentation, Bool, Vector{Int64}, Int64, NeuralPDE.Phi{Lux.Chain{NamedTuple{(:layer_1, :layer_2, :layer_3, :layer_4), Tuple{Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(identity), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}}}}, NamedTuple{(:layer_1, :layer_2, :layer_3, :layer_4), NTuple{4, NamedTuple{(), Tuple{}}}}}, typeof(additional_loss), Bool, Vector{Float64}}, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, typeof(SciMLBase.DEFAULT_OBSERVED_NO_TIME), Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing}, Optimization.var"#228#237"{Optimization.var"#227#236"{OptimizationFunction{true, Optimization.AutoZygote, NeuralPDE.var"#full_loss_function#308"{NeuralPDE.var"#null_nonadaptive_loss#115", Vector{NeuralPDE.var"#74#75"{NeuralPDE.var"#209#210"{RuntimeGeneratedFunctions.RuntimeGeneratedFunction{(:cord, Symbol("##θ#293"), :phi, :derivative, :integral, :u, :p), NeuralPDE.var"#_RGF_ModTag", NeuralPDE.var"#_RGF_ModTag", (0xad10bb64, 0x1d2657c2, 0xc9c67c1a, 0xbaf4020a, 0xc4a1861b), Expr}, NeuralPDE.var"#12#13", NeuralPDE.var"#269#276"{NeuralPDE.var"#269#270#277"{typeof(NeuralPDE.numeric_derivative)}, Dict{Symbol, Int64}, Dict{Symbol, Int64}, GridTraining{Float64}}, typeof(NeuralPDE.numeric_derivative), NeuralPDE.Phi{Lux.Chain{NamedTuple{(:layer_1, :layer_2, :layer_3, :layer_4), Tuple{Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(identity), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}}}}, NamedTuple{(:layer_1, :layer_2, :layer_3, :layer_4), NTuple{4, NamedTuple{(), Tuple{}}}}}, Vector{Float64}}, Matrix{Float64}}}, Vector{NeuralPDE.var"#74#75"{NeuralPDE.var"#209#210"{RuntimeGeneratedFunctions.RuntimeGeneratedFunction{(:cord, Symbol("##θ#293"), :phi, :derivative, :integral, :u, :p), NeuralPDE.var"#_RGF_ModTag", NeuralPDE.var"#_RGF_ModTag", (0x7cf59a19, 0x0162d753, 0x260a99e5, 0xc4bc113e, 0x070b6155), Expr}, NeuralPDE.var"#12#13", NeuralPDE.var"#269#276"{NeuralPDE.var"#269#270#277"{typeof(NeuralPDE.numeric_derivative)}, Dict{Symbol, Int64}, Dict{Symbol, Int64}, GridTraining{Float64}}, typeof(NeuralPDE.numeric_derivative), NeuralPDE.Phi{Lux.Chain{NamedTuple{(:layer_1, :layer_2, :layer_3, :layer_4), Tuple{Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(identity), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}}}}, NamedTuple{(:layer_1, :layer_2, :layer_3, :layer_4), NTuple{4, NamedTuple{(), Tuple{}}}}}, Vector{Float64}}, Matrix{Float64}}}, NeuralPDE.PINNRepresentation, Bool, Vector{Int64}, Int64, NeuralPDE.Phi{Lux.Chain{NamedTuple{(:layer_1, :layer_2, :layer_3, :layer_4), Tuple{Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(identity), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}}}}, NamedTuple{(:layer_1, :layer_2, :layer_3, :layer_4), NTuple{4, NamedTuple{(), Tuple{}}}}}, typeof(additional_loss), Bool, Vector{Float64}}, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, typeof(SciMLBase.DEFAULT_OBSERVED_NO_TIME), Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing}, Optimization.ReInitCache{ComponentArrays.ComponentVector{Float64, Vector{Float64}, Tuple{ComponentArrays.Axis{(depvar = ViewAxis(1:217, Axis(layer_1 = ViewAxis(1:64, Axis(weight = ViewAxis(1:56, ShapedAxis((8, 7), NamedTuple())), bias = ViewAxis(57:64, ShapedAxis((8, 1), NamedTuple())))), layer_2 = ViewAxis(65:136, Axis(weight = ViewAxis(1:64, ShapedAxis((8, 8), NamedTuple())), bias = ViewAxis(65:72, ShapedAxis((8, 1), NamedTuple())))), layer_3 = ViewAxis(137:208, Axis(weight = ViewAxis(1:64, ShapedAxis((8, 8), NamedTuple())), bias = ViewAxis(65:72, ShapedAxis((8, 1), NamedTuple())))), layer_4 = ViewAxis(209:217, Axis(weight = ViewAxis(1:8, ShapedAxis((1, 8), NamedTuple())), bias = ViewAxis(9:9, ShapedAxis((1, 1), NamedTuple())))))), p = 218:220)}}}, SciMLBase.NullParameters}}}, Optimization.var"#231#240"{Optimization.var"#227#236"{OptimizationFunction{true, Optimization.AutoZygote, NeuralPDE.var"#full_loss_function#308"{NeuralPDE.var"#null_nonadaptive_loss#115", Vector{NeuralPDE.var"#74#75"{NeuralPDE.var"#209#210"{RuntimeGeneratedFunctions.RuntimeGeneratedFunction{(:cord, Symbol("##θ#293"), :phi, :derivative, :integral, :u, :p), NeuralPDE.var"#_RGF_ModTag", NeuralPDE.var"#_RGF_ModTag", (0xad10bb64, 0x1d2657c2, 0xc9c67c1a, 0xbaf4020a, 0xc4a1861b), Expr}, NeuralPDE.var"#12#13", NeuralPDE.var"#269#276"{NeuralPDE.var"#269#270#277"{typeof(NeuralPDE.numeric_derivative)}, Dict{Symbol, Int64}, Dict{Symbol, Int64}, GridTraining{Float64}}, typeof(NeuralPDE.numeric_derivative), NeuralPDE.Phi{Lux.Chain{NamedTuple{(:layer_1, :layer_2, :layer_3, :layer_4), Tuple{Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(identity), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}}}}, NamedTuple{(:layer_1, :layer_2, :layer_3, :layer_4), NTuple{4, NamedTuple{(), Tuple{}}}}}, Vector{Float64}}, Matrix{Float64}}}, Vector{NeuralPDE.var"#74#75"{NeuralPDE.var"#209#210"{RuntimeGeneratedFunctions.RuntimeGeneratedFunction{(:cord, Symbol("##θ#293"), :phi, :derivative, :integral, :u, :p), NeuralPDE.var"#_RGF_ModTag", NeuralPDE.var"#_RGF_ModTag", (0x7cf59a19, 0x0162d753, 0x260a99e5, 0xc4bc113e, 0x070b6155), Expr}, NeuralPDE.var"#12#13", NeuralPDE.var"#269#276"{NeuralPDE.var"#269#270#277"{typeof(NeuralPDE.numeric_derivative)}, Dict{Symbol, Int64}, Dict{Symbol, Int64}, GridTraining{Float64}}, typeof(NeuralPDE.numeric_derivative), NeuralPDE.Phi{Lux.Chain{NamedTuple{(:layer_1, :layer_2, :layer_3, :layer_4), Tuple{Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(identity), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}}}}, NamedTuple{(:layer_1, :layer_2, :layer_3, :layer_4), NTuple{4, NamedTuple{(), Tuple{}}}}}, Vector{Float64}}, Matrix{Float64}}}, NeuralPDE.PINNRepresentation, Bool, Vector{Int64}, Int64, NeuralPDE.Phi{Lux.Chain{NamedTuple{(:layer_1, :layer_2, :layer_3, :layer_4), Tuple{Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(identity), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}}}}, NamedTuple{(:layer_1, :layer_2, :layer_3, :layer_4), NTuple{4, NamedTuple{(), Tuple{}}}}}, typeof(additional_loss), Bool, Vector{Float64}}, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, typeof(SciMLBase.DEFAULT_OBSERVED_NO_TIME), Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing}, Optimization.ReInitCache{ComponentArrays.ComponentVector{Float64, Vector{Float64}, Tuple{ComponentArrays.Axis{(depvar = ViewAxis(1:217, Axis(layer_1 = ViewAxis(1:64, Axis(weight = ViewAxis(1:56, ShapedAxis((8, 7), NamedTuple())), bias = ViewAxis(57:64, ShapedAxis((8, 1), NamedTuple())))), layer_2 = ViewAxis(65:136, Axis(weight = ViewAxis(1:64, ShapedAxis((8, 8), NamedTuple())), bias = ViewAxis(65:72, ShapedAxis((8, 1), NamedTuple())))), layer_3 = ViewAxis(137:208, Axis(weight = ViewAxis(1:64, ShapedAxis((8, 8), NamedTuple())), bias = ViewAxis(65:72, ShapedAxis((8, 1), NamedTuple())))), layer_4 = ViewAxis(209:217, Axis(weight = ViewAxis(1:8, ShapedAxis((1, 8), NamedTuple())), bias = ViewAxis(9:9, ShapedAxis((1, 1), NamedTuple())))))), p = 218:220)}}}, SciMLBase.NullParameters}}}, Optimization.var"#235#244", Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, typeof(SciMLBase.DEFAULT_OBSERVED_NO_TIME), Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing}, Optimization.ReInitCache{ComponentArrays.ComponentVector{Float64, Vector{Float64}, Tuple{ComponentArrays.Axis{(depvar = ViewAxis(1:217, Axis(layer_1 = ViewAxis(1:64, Axis(weight = ViewAxis(1:56, ShapedAxis((8, 7), NamedTuple())), bias = ViewAxis(57:64, ShapedAxis((8, 1), NamedTuple())))), layer_2 = ViewAxis(65:136, Axis(weight = ViewAxis(1:64, ShapedAxis((8, 8), NamedTuple())), bias = ViewAxis(65:72, ShapedAxis((8, 1), NamedTuple())))), layer_3 = ViewAxis(137:208, Axis(weight = ViewAxis(1:64, ShapedAxis((8, 8), NamedTuple())), bias = ViewAxis(65:72, ShapedAxis((8, 1), NamedTuple())))), layer_4 = ViewAxis(209:217, Axis(weight = ViewAxis(1:8, ShapedAxis((1, 8), NamedTuple())), bias = ViewAxis(9:9, ShapedAxis((1, 1), NamedTuple())))))), p = 218:220)}}}, SciMLBase.NullParameters}, Nothing, Nothing, Nothing, Nothing, Nothing, BFGS{LineSearches.InitialStatic{Float64}, LineSearches.HagerZhang{Float64, Base.RefValue{Bool}}, Nothing, Nothing, Flat}, Base.Iterators.Cycle{Tuple{Optimization.NullData}}, Bool, var"#3#4"}}})(G::ComponentArrays.ComponentVector{Float64, Vector{Float64}, Tuple{ComponentArrays.Axis{(depvar = ViewAxis(1:217, Axis(layer_1 = ViewAxis(1:64, Axis(weight = ViewAxis(1:56, ShapedAxis((8, 7), NamedTuple())), bias = ViewAxis(57:64, ShapedAxis((8, 1), NamedTuple())))), layer_2 = ViewAxis(65:136, Axis(weight = ViewAxis(1:64, ShapedAxis((8, 8), NamedTuple())), bias = ViewAxis(65:72, ShapedAxis((8, 1), NamedTuple())))), layer_3 = ViewAxis(137:208, Axis(weight = ViewAxis(1:64, ShapedAxis((8, 8), NamedTuple())), bias = ViewAxis(65:72, ShapedAxis((8, 1), NamedTuple())))), layer_4 = ViewAxis(209:217, Axis(weight = ViewAxis(1:8, ShapedAxis((1, 8), NamedTuple())), bias = ViewAxis(9:9, ShapedAxis((1, 1), NamedTuple())))))), p = 218:220)}}}, θ::ComponentArrays.ComponentVector{Float64, Vector{Float64}, Tuple{ComponentArrays.Axis{(depvar = ViewAxis(1:217, Axis(layer_1 = ViewAxis(1:64, Axis(weight = ViewAxis(1:56, ShapedAxis((8, 7), NamedTuple())), bias = ViewAxis(57:64, ShapedAxis((8, 1), NamedTuple())))), layer_2 = ViewAxis(65:136, Axis(weight = ViewAxis(1:64, ShapedAxis((8, 8), NamedTuple())), bias = ViewAxis(65:72, ShapedAxis((8, 1), NamedTuple())))), layer_3 = ViewAxis(137:208, Axis(weight = ViewAxis(1:64, ShapedAxis((8, 8), NamedTuple())), bias = ViewAxis(65:72, ShapedAxis((8, 1), NamedTuple())))), layer_4 = ViewAxis(209:217, Axis(weight = ViewAxis(1:8, ShapedAxis((1, 8), NamedTuple())), bias = ViewAxis(9:9, ShapedAxis((1, 1), NamedTuple())))))), p = 218:220)}}})
    @ OptimizationOptimJL C:\Users\loren\.julia\packages\OptimizationOptimJL\uRfW9\src\OptimizationOptimJL.jl:200
 [55] value_gradient!!(obj::TwiceDifferentiable{Float64, ComponentArrays.ComponentVector{Float64, Vector{Float64}, Tuple{ComponentArrays.Axis{(depvar = ViewAxis(1:217, Axis(layer_1 = ViewAxis(1:64, Axis(weight = ViewAxis(1:56, ShapedAxis((8, 7), NamedTuple())), bias = ViewAxis(57:64, ShapedAxis((8, 1), NamedTuple())))), layer_2 = ViewAxis(65:136, Axis(weight = ViewAxis(1:64, ShapedAxis((8, 8), NamedTuple())), bias = ViewAxis(65:72, ShapedAxis((8, 1), NamedTuple())))), layer_3 = ViewAxis(137:208, Axis(weight = ViewAxis(1:64, ShapedAxis((8, 8), NamedTuple())), bias = ViewAxis(65:72, ShapedAxis((8, 1), NamedTuple())))), layer_4 = ViewAxis(209:217, Axis(weight = ViewAxis(1:8, ShapedAxis((1, 8), NamedTuple())), bias = ViewAxis(9:9, ShapedAxis((1, 1), NamedTuple())))))), p = 218:220)}}}, ComponentArrays.ComponentMatrix{Float64, Matrix{Float64}, Tuple{ComponentArrays.Axis{(depvar = ViewAxis(1:217, Axis(layer_1 = ViewAxis(1:64, Axis(weight = ViewAxis(1:56, ShapedAxis((8, 7), NamedTuple())), bias = ViewAxis(57:64, ShapedAxis((8, 1), NamedTuple())))), layer_2 = ViewAxis(65:136, Axis(weight = ViewAxis(1:64, ShapedAxis((8, 8), NamedTuple())), bias = ViewAxis(65:72, ShapedAxis((8, 1), NamedTuple())))), layer_3 = ViewAxis(137:208, Axis(weight = ViewAxis(1:64, ShapedAxis((8, 8), NamedTuple())), bias = ViewAxis(65:72, ShapedAxis((8, 1), NamedTuple())))), layer_4 = ViewAxis(209:217, Axis(weight = ViewAxis(1:8, ShapedAxis((1, 8), NamedTuple())), bias = ViewAxis(9:9, ShapedAxis((1, 1), NamedTuple())))))), p = 218:220)}, ComponentArrays.Axis{(depvar = ViewAxis(1:217, Axis(layer_1 = ViewAxis(1:64, Axis(weight = ViewAxis(1:56, ShapedAxis((8, 7), NamedTuple())), bias = ViewAxis(57:64, ShapedAxis((8, 1), NamedTuple())))), layer_2 = ViewAxis(65:136, Axis(weight = ViewAxis(1:64, ShapedAxis((8, 8), NamedTuple())), bias = ViewAxis(65:72, ShapedAxis((8, 1), NamedTuple())))), layer_3 = ViewAxis(137:208, Axis(weight = ViewAxis(1:64, ShapedAxis((8, 8), NamedTuple())), bias = ViewAxis(65:72, ShapedAxis((8, 1), NamedTuple())))), layer_4 = ViewAxis(209:217, Axis(weight = ViewAxis(1:8, ShapedAxis((1, 8), NamedTuple())), bias = ViewAxis(9:9, ShapedAxis((1, 1), NamedTuple())))))), p = 218:220)}}}, ComponentArrays.ComponentVector{Float64, Vector{Float64}, Tuple{ComponentArrays.Axis{(depvar = ViewAxis(1:217, Axis(layer_1 = ViewAxis(1:64, Axis(weight = ViewAxis(1:56, ShapedAxis((8, 7), NamedTuple())), bias = ViewAxis(57:64, ShapedAxis((8, 1), NamedTuple())))), layer_2 = ViewAxis(65:136, Axis(weight = ViewAxis(1:64, ShapedAxis((8, 8), NamedTuple())), bias = ViewAxis(65:72, ShapedAxis((8, 1), NamedTuple())))), layer_3 = ViewAxis(137:208, Axis(weight = ViewAxis(1:64, ShapedAxis((8, 8), NamedTuple())), bias = ViewAxis(65:72, ShapedAxis((8, 1), NamedTuple())))), layer_4 = ViewAxis(209:217, Axis(weight = ViewAxis(1:8, ShapedAxis((1, 8), NamedTuple())), bias = ViewAxis(9:9, ShapedAxis((1, 1), NamedTuple())))))), p = 218:220)}}}}, x::ComponentArrays.ComponentVector{Float64, Vector{Float64}, Tuple{ComponentArrays.Axis{(depvar = ViewAxis(1:217, Axis(layer_1 = ViewAxis(1:64, Axis(weight = ViewAxis(1:56, ShapedAxis((8, 7), NamedTuple())), bias = ViewAxis(57:64, ShapedAxis((8, 1), NamedTuple())))), layer_2 = ViewAxis(65:136, Axis(weight = ViewAxis(1:64, ShapedAxis((8, 8), NamedTuple())), bias = ViewAxis(65:72, ShapedAxis((8, 1), NamedTuple())))), layer_3 = ViewAxis(137:208, Axis(weight = ViewAxis(1:64, ShapedAxis((8, 8), NamedTuple())), bias = ViewAxis(65:72, ShapedAxis((8, 1), NamedTuple())))), layer_4 = ViewAxis(209:217, Axis(weight = ViewAxis(1:8, ShapedAxis((1, 8), NamedTuple())), bias = ViewAxis(9:9, ShapedAxis((1, 1), NamedTuple())))))), p = 218:220)}}})
    @ NLSolversBase C:\Users\loren\.julia\packages\NLSolversBase\kavn7\src\interface.jl:82
 [56] initial_state(method::BFGS{LineSearches.InitialStatic{Float64}, LineSearches.HagerZhang{Float64, Base.RefValue{Bool}}, Nothing, Nothing, Flat}, options::Optim.Options{Float64, OptimizationOptimJL.var"#_cb#14"{OptimizationOptimJL.OptimJLOptimizationCache{OptimizationFunction{false, Optimization.AutoZygote, OptimizationFunction{true, Optimization.AutoZygote, NeuralPDE.var"#full_loss_function#308"{NeuralPDE.var"#null_nonadaptive_loss#115", Vector{NeuralPDE.var"#74#75"{NeuralPDE.var"#209#210"{RuntimeGeneratedFunctions.RuntimeGeneratedFunction{(:cord, Symbol("##θ#293"), :phi, :derivative, :integral, :u, :p), NeuralPDE.var"#_RGF_ModTag", NeuralPDE.var"#_RGF_ModTag", (0xad10bb64, 0x1d2657c2, 0xc9c67c1a, 0xbaf4020a, 0xc4a1861b), Expr}, NeuralPDE.var"#12#13", NeuralPDE.var"#269#276"{NeuralPDE.var"#269#270#277"{typeof(NeuralPDE.numeric_derivative)}, Dict{Symbol, Int64}, Dict{Symbol, Int64}, GridTraining{Float64}}, typeof(NeuralPDE.numeric_derivative), NeuralPDE.Phi{Lux.Chain{NamedTuple{(:layer_1, :layer_2, :layer_3, :layer_4), Tuple{Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(identity), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}}}}, NamedTuple{(:layer_1, :layer_2, :layer_3, :layer_4), NTuple{4, NamedTuple{(), Tuple{}}}}}, Vector{Float64}}, Matrix{Float64}}}, Vector{NeuralPDE.var"#74#75"{NeuralPDE.var"#209#210"{RuntimeGeneratedFunctions.RuntimeGeneratedFunction{(:cord, Symbol("##θ#293"), :phi, :derivative, :integral, :u, :p), NeuralPDE.var"#_RGF_ModTag", NeuralPDE.var"#_RGF_ModTag", (0x7cf59a19, 0x0162d753, 0x260a99e5, 0xc4bc113e, 0x070b6155), Expr}, NeuralPDE.var"#12#13", NeuralPDE.var"#269#276"{NeuralPDE.var"#269#270#277"{typeof(NeuralPDE.numeric_derivative)}, Dict{Symbol, Int64}, Dict{Symbol, Int64}, GridTraining{Float64}}, typeof(NeuralPDE.numeric_derivative), NeuralPDE.Phi{Lux.Chain{NamedTuple{(:layer_1, :layer_2, :layer_3, :layer_4), Tuple{Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(identity), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}}}}, NamedTuple{(:layer_1, :layer_2, :layer_3, :layer_4), NTuple{4, NamedTuple{(), Tuple{}}}}}, Vector{Float64}}, Matrix{Float64}}}, NeuralPDE.PINNRepresentation, Bool, Vector{Int64}, Int64, NeuralPDE.Phi{Lux.Chain{NamedTuple{(:layer_1, :layer_2, :layer_3, :layer_4), Tuple{Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(identity), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}}}}, NamedTuple{(:layer_1, :layer_2, :layer_3, :layer_4), NTuple{4, NamedTuple{(), Tuple{}}}}}, typeof(additional_loss), Bool, Vector{Float64}}, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, typeof(SciMLBase.DEFAULT_OBSERVED_NO_TIME), Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing}, Optimization.var"#228#237"{Optimization.var"#227#236"{OptimizationFunction{true, Optimization.AutoZygote, NeuralPDE.var"#full_loss_function#308"{NeuralPDE.var"#null_nonadaptive_loss#115", Vector{NeuralPDE.var"#74#75"{NeuralPDE.var"#209#210"{RuntimeGeneratedFunctions.RuntimeGeneratedFunction{(:cord, Symbol("##θ#293"), :phi, :derivative, :integral, :u, :p), NeuralPDE.var"#_RGF_ModTag", NeuralPDE.var"#_RGF_ModTag", (0xad10bb64, 0x1d2657c2, 0xc9c67c1a, 0xbaf4020a, 0xc4a1861b), Expr}, NeuralPDE.var"#12#13", NeuralPDE.var"#269#276"{NeuralPDE.var"#269#270#277"{typeof(NeuralPDE.numeric_derivative)}, Dict{Symbol, Int64}, Dict{Symbol, Int64}, GridTraining{Float64}}, typeof(NeuralPDE.numeric_derivative), NeuralPDE.Phi{Lux.Chain{NamedTuple{(:layer_1, :layer_2, :layer_3, :layer_4), Tuple{Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(identity), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}}}}, NamedTuple{(:layer_1, :layer_2, :layer_3, :layer_4), NTuple{4, NamedTuple{(), Tuple{}}}}}, Vector{Float64}}, Matrix{Float64}}}, Vector{NeuralPDE.var"#74#75"{NeuralPDE.var"#209#210"{RuntimeGeneratedFunctions.RuntimeGeneratedFunction{(:cord, Symbol("##θ#293"), :phi, :derivative, :integral, :u, :p), NeuralPDE.var"#_RGF_ModTag", NeuralPDE.var"#_RGF_ModTag", (0x7cf59a19, 0x0162d753, 0x260a99e5, 0xc4bc113e, 0x070b6155), Expr}, NeuralPDE.var"#12#13", NeuralPDE.var"#269#276"{NeuralPDE.var"#269#270#277"{typeof(NeuralPDE.numeric_derivative)}, Dict{Symbol, Int64}, Dict{Symbol, Int64}, GridTraining{Float64}}, typeof(NeuralPDE.numeric_derivative), NeuralPDE.Phi{Lux.Chain{NamedTuple{(:layer_1, :layer_2, :layer_3, :layer_4), Tuple{Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(identity), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}}}}, NamedTuple{(:layer_1, :layer_2, :layer_3, :layer_4), NTuple{4, NamedTuple{(), Tuple{}}}}}, Vector{Float64}}, Matrix{Float64}}}, NeuralPDE.PINNRepresentation, Bool, Vector{Int64}, Int64, NeuralPDE.Phi{Lux.Chain{NamedTuple{(:layer_1, :layer_2, :layer_3, :layer_4), Tuple{Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(identity), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}}}}, NamedTuple{(:layer_1, :layer_2, :layer_3, :layer_4), NTuple{4, NamedTuple{(), Tuple{}}}}}, typeof(additional_loss), Bool, Vector{Float64}}, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, typeof(SciMLBase.DEFAULT_OBSERVED_NO_TIME), Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing}, Optimization.ReInitCache{ComponentArrays.ComponentVector{Float64, Vector{Float64}, Tuple{ComponentArrays.Axis{(depvar = ViewAxis(1:217, Axis(layer_1 = ViewAxis(1:64, Axis(weight = ViewAxis(1:56, ShapedAxis((8, 7), NamedTuple())), bias = ViewAxis(57:64, ShapedAxis((8, 1), NamedTuple())))), layer_2 = ViewAxis(65:136, Axis(weight = ViewAxis(1:64, ShapedAxis((8, 8), NamedTuple())), bias = ViewAxis(65:72, ShapedAxis((8, 1), NamedTuple())))), layer_3 = ViewAxis(137:208, Axis(weight = ViewAxis(1:64, ShapedAxis((8, 8), NamedTuple())), bias = ViewAxis(65:72, ShapedAxis((8, 1), NamedTuple())))), layer_4 = ViewAxis(209:217, Axis(weight = ViewAxis(1:8, ShapedAxis((1, 8), NamedTuple())), bias = ViewAxis(9:9, ShapedAxis((1, 1), NamedTuple())))))), p = 218:220)}}}, SciMLBase.NullParameters}}}, Optimization.var"#231#240"{Optimization.var"#227#236"{OptimizationFunction{true, Optimization.AutoZygote, NeuralPDE.var"#full_loss_function#308"{NeuralPDE.var"#null_nonadaptive_loss#115", Vector{NeuralPDE.var"#74#75"{NeuralPDE.var"#209#210"{RuntimeGeneratedFunctions.RuntimeGeneratedFunction{(:cord, Symbol("##θ#293"), :phi, :derivative, :integral, :u, :p), NeuralPDE.var"#_RGF_ModTag", NeuralPDE.var"#_RGF_ModTag", (0xad10bb64, 0x1d2657c2, 0xc9c67c1a, 0xbaf4020a, 0xc4a1861b), Expr}, NeuralPDE.var"#12#13", NeuralPDE.var"#269#276"{NeuralPDE.var"#269#270#277"{typeof(NeuralPDE.numeric_derivative)}, Dict{Symbol, Int64}, Dict{Symbol, Int64}, GridTraining{Float64}}, typeof(NeuralPDE.numeric_derivative), NeuralPDE.Phi{Lux.Chain{NamedTuple{(:layer_1, :layer_2, :layer_3, :layer_4), Tuple{Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(identity), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}}}}, NamedTuple{(:layer_1, :layer_2, :layer_3, :layer_4), NTuple{4, NamedTuple{(), Tuple{}}}}}, Vector{Float64}}, Matrix{Float64}}}, Vector{NeuralPDE.var"#74#75"{NeuralPDE.var"#209#210"{RuntimeGeneratedFunctions.RuntimeGeneratedFunction{(:cord, Symbol("##θ#293"), :phi, :derivative, :integral, :u, :p), NeuralPDE.var"#_RGF_ModTag", NeuralPDE.var"#_RGF_ModTag", (0x7cf59a19, 0x0162d753, 0x260a99e5, 0xc4bc113e, 0x070b6155), Expr}, NeuralPDE.var"#12#13", NeuralPDE.var"#269#276"{NeuralPDE.var"#269#270#277"{typeof(NeuralPDE.numeric_derivative)}, Dict{Symbol, Int64}, Dict{Symbol, Int64}, GridTraining{Float64}}, typeof(NeuralPDE.numeric_derivative), NeuralPDE.Phi{Lux.Chain{NamedTuple{(:layer_1, :layer_2, :layer_3, :layer_4), Tuple{Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(identity), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}}}}, NamedTuple{(:layer_1, :layer_2, :layer_3, :layer_4), NTuple{4, NamedTuple{(), Tuple{}}}}}, Vector{Float64}}, Matrix{Float64}}}, NeuralPDE.PINNRepresentation, Bool, Vector{Int64}, Int64, NeuralPDE.Phi{Lux.Chain{NamedTuple{(:layer_1, :layer_2, :layer_3, :layer_4), Tuple{Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(identity), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}}}}, NamedTuple{(:layer_1, :layer_2, :layer_3, :layer_4), NTuple{4, NamedTuple{(), Tuple{}}}}}, typeof(additional_loss), Bool, Vector{Float64}}, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, typeof(SciMLBase.DEFAULT_OBSERVED_NO_TIME), Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing}, Optimization.ReInitCache{ComponentArrays.ComponentVector{Float64, Vector{Float64}, Tuple{ComponentArrays.Axis{(depvar = ViewAxis(1:217, Axis(layer_1 = ViewAxis(1:64, Axis(weight = ViewAxis(1:56, ShapedAxis((8, 7), NamedTuple())), bias = ViewAxis(57:64, ShapedAxis((8, 1), NamedTuple())))), layer_2 = ViewAxis(65:136, Axis(weight = ViewAxis(1:64, ShapedAxis((8, 8), NamedTuple())), bias = ViewAxis(65:72, ShapedAxis((8, 1), NamedTuple())))), layer_3 = ViewAxis(137:208, Axis(weight = ViewAxis(1:64, ShapedAxis((8, 8), NamedTuple())), bias = ViewAxis(65:72, ShapedAxis((8, 1), NamedTuple())))), layer_4 = ViewAxis(209:217, Axis(weight = ViewAxis(1:8, ShapedAxis((1, 8), NamedTuple())), bias = ViewAxis(9:9, ShapedAxis((1, 1), NamedTuple())))))), p = 218:220)}}}, SciMLBase.NullParameters}}}, Optimization.var"#235#244", Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, typeof(SciMLBase.DEFAULT_OBSERVED_NO_TIME), Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing}, Optimization.ReInitCache{ComponentArrays.ComponentVector{Float64, Vector{Float64}, Tuple{ComponentArrays.Axis{(depvar = ViewAxis(1:217, Axis(layer_1 = ViewAxis(1:64, Axis(weight = ViewAxis(1:56, ShapedAxis((8, 7), NamedTuple())), bias = ViewAxis(57:64, ShapedAxis((8, 1), NamedTuple())))), layer_2 = ViewAxis(65:136, Axis(weight = ViewAxis(1:64, ShapedAxis((8, 8), NamedTuple())), bias = ViewAxis(65:72, ShapedAxis((8, 1), NamedTuple())))), layer_3 = ViewAxis(137:208, Axis(weight = ViewAxis(1:64, ShapedAxis((8, 8), NamedTuple())), bias = ViewAxis(65:72, ShapedAxis((8, 1), NamedTuple())))), layer_4 = ViewAxis(209:217, Axis(weight = ViewAxis(1:8, ShapedAxis((1, 8), NamedTuple())), bias = ViewAxis(9:9, ShapedAxis((1, 1), NamedTuple())))))), p = 218:220)}}}, SciMLBase.NullParameters}, Nothing, Nothing, Nothing, Nothing, Nothing, BFGS{LineSearches.InitialStatic{Float64}, LineSearches.HagerZhang{Float64, Base.RefValue{Bool}}, Nothing, Nothing, Flat}, Base.Iterators.Cycle{Tuple{Optimization.NullData}}, Bool, var"#3#4"}}}, d::TwiceDifferentiable{Float64, ComponentArrays.ComponentVector{Float64, Vector{Float64}, Tuple{ComponentArrays.Axis{(depvar = ViewAxis(1:217, Axis(layer_1 = ViewAxis(1:64, Axis(weight = ViewAxis(1:56, ShapedAxis((8, 7), NamedTuple())), bias = ViewAxis(57:64, ShapedAxis((8, 1), NamedTuple())))), layer_2 = ViewAxis(65:136, Axis(weight = ViewAxis(1:64, ShapedAxis((8, 8), NamedTuple())), bias = ViewAxis(65:72, ShapedAxis((8, 1), NamedTuple())))), layer_3 = ViewAxis(137:208, Axis(weight = ViewAxis(1:64, ShapedAxis((8, 8), NamedTuple())), bias = ViewAxis(65:72, ShapedAxis((8, 1), NamedTuple())))), layer_4 = ViewAxis(209:217, Axis(weight = ViewAxis(1:8, ShapedAxis((1, 8), NamedTuple())), bias = ViewAxis(9:9, ShapedAxis((1, 1), NamedTuple())))))), p = 218:220)}}}, ComponentArrays.ComponentMatrix{Float64, Matrix{Float64}, Tuple{ComponentArrays.Axis{(depvar = ViewAxis(1:217, Axis(layer_1 = ViewAxis(1:64, Axis(weight = ViewAxis(1:56, ShapedAxis((8, 7), NamedTuple())), bias = ViewAxis(57:64, ShapedAxis((8, 1), NamedTuple())))), layer_2 = ViewAxis(65:136, Axis(weight = ViewAxis(1:64, ShapedAxis((8, 8), NamedTuple())), bias = ViewAxis(65:72, ShapedAxis((8, 1), NamedTuple())))), layer_3 = ViewAxis(137:208, Axis(weight = ViewAxis(1:64, ShapedAxis((8, 8), NamedTuple())), bias = ViewAxis(65:72, ShapedAxis((8, 1), NamedTuple())))), layer_4 = ViewAxis(209:217, Axis(weight = ViewAxis(1:8, ShapedAxis((1, 8), NamedTuple())), bias = ViewAxis(9:9, ShapedAxis((1, 1), NamedTuple())))))), p = 218:220)}, ComponentArrays.Axis{(depvar = ViewAxis(1:217, Axis(layer_1 = ViewAxis(1:64, Axis(weight = ViewAxis(1:56, ShapedAxis((8, 7), NamedTuple())), bias = ViewAxis(57:64, ShapedAxis((8, 1), NamedTuple())))), layer_2 = ViewAxis(65:136, Axis(weight = ViewAxis(1:64, ShapedAxis((8, 8), NamedTuple())), bias = ViewAxis(65:72, ShapedAxis((8, 1), NamedTuple())))), layer_3 = ViewAxis(137:208, Axis(weight = ViewAxis(1:64, ShapedAxis((8, 8), NamedTuple())), bias = ViewAxis(65:72, ShapedAxis((8, 1), NamedTuple())))), layer_4 = ViewAxis(209:217, Axis(weight = ViewAxis(1:8, ShapedAxis((1, 8), NamedTuple())), bias = ViewAxis(9:9, ShapedAxis((1, 1), NamedTuple())))))), p = 218:220)}}}, ComponentArrays.ComponentVector{Float64, Vector{Float64}, Tuple{ComponentArrays.Axis{(depvar = ViewAxis(1:217, Axis(layer_1 = ViewAxis(1:64, Axis(weight = ViewAxis(1:56, ShapedAxis((8, 7), NamedTuple())), bias = ViewAxis(57:64, ShapedAxis((8, 1), NamedTuple())))), layer_2 = ViewAxis(65:136, Axis(weight = ViewAxis(1:64, ShapedAxis((8, 8), NamedTuple())), bias = ViewAxis(65:72, ShapedAxis((8, 1), NamedTuple())))), layer_3 = ViewAxis(137:208, Axis(weight = ViewAxis(1:64, ShapedAxis((8, 8), NamedTuple())), bias = ViewAxis(65:72, ShapedAxis((8, 1), NamedTuple())))), layer_4 = ViewAxis(209:217, Axis(weight = ViewAxis(1:8, ShapedAxis((1, 8), NamedTuple())), bias = ViewAxis(9:9, ShapedAxis((1, 1), NamedTuple())))))), p = 218:220)}}}}, initial_x::ComponentArrays.ComponentVector{Float64, Vector{Float64}, Tuple{ComponentArrays.Axis{(depvar = ViewAxis(1:217, Axis(layer_1 = ViewAxis(1:64, Axis(weight = ViewAxis(1:56, ShapedAxis((8, 7), NamedTuple())), bias = ViewAxis(57:64, ShapedAxis((8, 1), NamedTuple())))), layer_2 = ViewAxis(65:136, Axis(weight = ViewAxis(1:64, ShapedAxis((8, 8), NamedTuple())), bias = ViewAxis(65:72, ShapedAxis((8, 1), NamedTuple())))), layer_3 = ViewAxis(137:208, Axis(weight = ViewAxis(1:64, ShapedAxis((8, 8), NamedTuple())), bias = ViewAxis(65:72, ShapedAxis((8, 1), NamedTuple())))), layer_4 = ViewAxis(209:217, Axis(weight = ViewAxis(1:8, ShapedAxis((1, 8), NamedTuple())), bias = ViewAxis(9:9, ShapedAxis((1, 1), NamedTuple())))))), p = 218:220)}}})
    @ Optim C:\Users\loren\.julia\packages\Optim\V8ZEC\src\multivariate\solvers\first_order\bfgs.jl:94
 [57] optimize(d::TwiceDifferentiable{Float64, ComponentArrays.ComponentVector{Float64, Vector{Float64}, Tuple{ComponentArrays.Axis{(depvar = ViewAxis(1:217, Axis(layer_1 = ViewAxis(1:64, Axis(weight = ViewAxis(1:56, ShapedAxis((8, 7), NamedTuple())), bias = ViewAxis(57:64, ShapedAxis((8, 1), NamedTuple())))), layer_2 = ViewAxis(65:136, Axis(weight = ViewAxis(1:64, ShapedAxis((8, 8), NamedTuple())), bias = ViewAxis(65:72, ShapedAxis((8, 1), NamedTuple())))), layer_3 = ViewAxis(137:208, Axis(weight = ViewAxis(1:64, ShapedAxis((8, 8), NamedTuple())), bias = ViewAxis(65:72, ShapedAxis((8, 1), NamedTuple())))), layer_4 = ViewAxis(209:217, Axis(weight = ViewAxis(1:8, ShapedAxis((1, 8), NamedTuple())), bias = ViewAxis(9:9, ShapedAxis((1, 1), NamedTuple())))))), p = 218:220)}}}, ComponentArrays.ComponentMatrix{Float64, Matrix{Float64}, Tuple{ComponentArrays.Axis{(depvar = ViewAxis(1:217, Axis(layer_1 = ViewAxis(1:64, Axis(weight = ViewAxis(1:56, ShapedAxis((8, 7), NamedTuple())), bias = ViewAxis(57:64, ShapedAxis((8, 1), NamedTuple())))), layer_2 = ViewAxis(65:136, Axis(weight = ViewAxis(1:64, ShapedAxis((8, 8), NamedTuple())), bias = ViewAxis(65:72, ShapedAxis((8, 1), NamedTuple())))), layer_3 = ViewAxis(137:208, Axis(weight = ViewAxis(1:64, ShapedAxis((8, 8), NamedTuple())), bias = ViewAxis(65:72, ShapedAxis((8, 1), NamedTuple())))), layer_4 = ViewAxis(209:217, Axis(weight = ViewAxis(1:8, ShapedAxis((1, 8), NamedTuple())), bias = ViewAxis(9:9, ShapedAxis((1, 1), NamedTuple())))))), p = 218:220)}, ComponentArrays.Axis{(depvar = ViewAxis(1:217, Axis(layer_1 = ViewAxis(1:64, Axis(weight = ViewAxis(1:56, ShapedAxis((8, 7), NamedTuple())), bias = ViewAxis(57:64, ShapedAxis((8, 1), NamedTuple())))), layer_2 = ViewAxis(65:136, Axis(weight = ViewAxis(1:64, ShapedAxis((8, 8), NamedTuple())), bias = ViewAxis(65:72, ShapedAxis((8, 1), NamedTuple())))), layer_3 = ViewAxis(137:208, Axis(weight = ViewAxis(1:64, ShapedAxis((8, 8), NamedTuple())), bias = ViewAxis(65:72, ShapedAxis((8, 1), NamedTuple())))), layer_4 = ViewAxis(209:217, Axis(weight = ViewAxis(1:8, ShapedAxis((1, 8), NamedTuple())), bias = ViewAxis(9:9, ShapedAxis((1, 1), NamedTuple())))))), p = 218:220)}}}, ComponentArrays.ComponentVector{Float64, Vector{Float64}, Tuple{ComponentArrays.Axis{(depvar = ViewAxis(1:217, Axis(layer_1 = ViewAxis(1:64, Axis(weight = ViewAxis(1:56, ShapedAxis((8, 7), NamedTuple())), bias = ViewAxis(57:64, ShapedAxis((8, 1), NamedTuple())))), layer_2 = ViewAxis(65:136, Axis(weight = ViewAxis(1:64, ShapedAxis((8, 8), NamedTuple())), bias = ViewAxis(65:72, ShapedAxis((8, 1), NamedTuple())))), layer_3 = ViewAxis(137:208, Axis(weight = ViewAxis(1:64, ShapedAxis((8, 8), NamedTuple())), bias = ViewAxis(65:72, ShapedAxis((8, 1), NamedTuple())))), layer_4 = ViewAxis(209:217, Axis(weight = ViewAxis(1:8, ShapedAxis((1, 8), NamedTuple())), bias = ViewAxis(9:9, ShapedAxis((1, 1), NamedTuple())))))), p = 218:220)}}}}, initial_x::ComponentArrays.ComponentVector{Float64, Vector{Float64}, Tuple{ComponentArrays.Axis{(depvar = ViewAxis(1:217, Axis(layer_1 = ViewAxis(1:64, Axis(weight = ViewAxis(1:56, ShapedAxis((8, 7), NamedTuple())), bias = ViewAxis(57:64, ShapedAxis((8, 1), NamedTuple())))), layer_2 = ViewAxis(65:136, Axis(weight = ViewAxis(1:64, ShapedAxis((8, 8), NamedTuple())), bias = ViewAxis(65:72, ShapedAxis((8, 1), NamedTuple())))), layer_3 = ViewAxis(137:208, Axis(weight = ViewAxis(1:64, ShapedAxis((8, 8), NamedTuple())), bias = ViewAxis(65:72, ShapedAxis((8, 1), NamedTuple())))), layer_4 = ViewAxis(209:217, Axis(weight = ViewAxis(1:8, ShapedAxis((1, 8), NamedTuple())), bias = ViewAxis(9:9, ShapedAxis((1, 1), NamedTuple())))))), p = 218:220)}}}, method::BFGS{LineSearches.InitialStatic{Float64}, LineSearches.HagerZhang{Float64, Base.RefValue{Bool}}, Nothing, Nothing, Flat}, options::Optim.Options{Float64, OptimizationOptimJL.var"#_cb#14"{OptimizationOptimJL.OptimJLOptimizationCache{OptimizationFunction{false, Optimization.AutoZygote, OptimizationFunction{true, Optimization.AutoZygote, NeuralPDE.var"#full_loss_function#308"{NeuralPDE.var"#null_nonadaptive_loss#115", Vector{NeuralPDE.var"#74#75"{NeuralPDE.var"#209#210"{RuntimeGeneratedFunctions.RuntimeGeneratedFunction{(:cord, Symbol("##θ#293"), :phi, :derivative, :integral, :u, :p), NeuralPDE.var"#_RGF_ModTag", NeuralPDE.var"#_RGF_ModTag", (0xad10bb64, 0x1d2657c2, 0xc9c67c1a, 0xbaf4020a, 0xc4a1861b), Expr}, NeuralPDE.var"#12#13", NeuralPDE.var"#269#276"{NeuralPDE.var"#269#270#277"{typeof(NeuralPDE.numeric_derivative)}, Dict{Symbol, Int64}, Dict{Symbol, Int64}, GridTraining{Float64}}, typeof(NeuralPDE.numeric_derivative), NeuralPDE.Phi{Lux.Chain{NamedTuple{(:layer_1, :layer_2, :layer_3, :layer_4), Tuple{Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(identity), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}}}}, NamedTuple{(:layer_1, :layer_2, :layer_3, :layer_4), NTuple{4, NamedTuple{(), Tuple{}}}}}, Vector{Float64}}, Matrix{Float64}}}, Vector{NeuralPDE.var"#74#75"{NeuralPDE.var"#209#210"{RuntimeGeneratedFunctions.RuntimeGeneratedFunction{(:cord, Symbol("##θ#293"), :phi, :derivative, :integral, :u, :p), NeuralPDE.var"#_RGF_ModTag", NeuralPDE.var"#_RGF_ModTag", (0x7cf59a19, 0x0162d753, 0x260a99e5, 0xc4bc113e, 0x070b6155), Expr}, NeuralPDE.var"#12#13", NeuralPDE.var"#269#276"{NeuralPDE.var"#269#270#277"{typeof(NeuralPDE.numeric_derivative)}, Dict{Symbol, Int64}, Dict{Symbol, Int64}, GridTraining{Float64}}, typeof(NeuralPDE.numeric_derivative), NeuralPDE.Phi{Lux.Chain{NamedTuple{(:layer_1, :layer_2, :layer_3, :layer_4), Tuple{Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(identity), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}}}}, NamedTuple{(:layer_1, :layer_2, :layer_3, :layer_4), NTuple{4, NamedTuple{(), Tuple{}}}}}, Vector{Float64}}, Matrix{Float64}}}, NeuralPDE.PINNRepresentation, Bool, Vector{Int64}, Int64, NeuralPDE.Phi{Lux.Chain{NamedTuple{(:layer_1, :layer_2, :layer_3, :layer_4), Tuple{Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(identity), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}}}}, NamedTuple{(:layer_1, :layer_2, :layer_3, :layer_4), NTuple{4, NamedTuple{(), Tuple{}}}}}, typeof(additional_loss), Bool, Vector{Float64}}, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, typeof(SciMLBase.DEFAULT_OBSERVED_NO_TIME), Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing}, Optimization.var"#228#237"{Optimization.var"#227#236"{OptimizationFunction{true, Optimization.AutoZygote, NeuralPDE.var"#full_loss_function#308"{NeuralPDE.var"#null_nonadaptive_loss#115", Vector{NeuralPDE.var"#74#75"{NeuralPDE.var"#209#210"{RuntimeGeneratedFunctions.RuntimeGeneratedFunction{(:cord, Symbol("##θ#293"), :phi, :derivative, :integral, :u, :p), NeuralPDE.var"#_RGF_ModTag", NeuralPDE.var"#_RGF_ModTag", (0xad10bb64, 0x1d2657c2, 0xc9c67c1a, 0xbaf4020a, 0xc4a1861b), Expr}, NeuralPDE.var"#12#13", NeuralPDE.var"#269#276"{NeuralPDE.var"#269#270#277"{typeof(NeuralPDE.numeric_derivative)}, Dict{Symbol, Int64}, Dict{Symbol, Int64}, GridTraining{Float64}}, typeof(NeuralPDE.numeric_derivative), NeuralPDE.Phi{Lux.Chain{NamedTuple{(:layer_1, :layer_2, :layer_3, :layer_4), Tuple{Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(identity), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}}}}, NamedTuple{(:layer_1, :layer_2, :layer_3, :layer_4), NTuple{4, NamedTuple{(), Tuple{}}}}}, Vector{Float64}}, Matrix{Float64}}}, Vector{NeuralPDE.var"#74#75"{NeuralPDE.var"#209#210"{RuntimeGeneratedFunctions.RuntimeGeneratedFunction{(:cord, Symbol("##θ#293"), :phi, :derivative, :integral, :u, :p), NeuralPDE.var"#_RGF_ModTag", NeuralPDE.var"#_RGF_ModTag", (0x7cf59a19, 0x0162d753, 0x260a99e5, 0xc4bc113e, 0x070b6155), Expr}, NeuralPDE.var"#12#13", NeuralPDE.var"#269#276"{NeuralPDE.var"#269#270#277"{typeof(NeuralPDE.numeric_derivative)}, Dict{Symbol, Int64}, Dict{Symbol, Int64}, GridTraining{Float64}}, typeof(NeuralPDE.numeric_derivative), NeuralPDE.Phi{Lux.Chain{NamedTuple{(:layer_1, :layer_2, :layer_3, :layer_4), Tuple{Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(identity), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}}}}, NamedTuple{(:layer_1, :layer_2, :layer_3, :layer_4), NTuple{4, NamedTuple{(), Tuple{}}}}}, Vector{Float64}}, Matrix{Float64}}}, NeuralPDE.PINNRepresentation, Bool, Vector{Int64}, Int64, NeuralPDE.Phi{Lux.Chain{NamedTuple{(:layer_1, :layer_2, :layer_3, :layer_4), Tuple{Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(identity), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}}}}, NamedTuple{(:layer_1, :layer_2, :layer_3, :layer_4), NTuple{4, NamedTuple{(), Tuple{}}}}}, typeof(additional_loss), Bool, Vector{Float64}}, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, typeof(SciMLBase.DEFAULT_OBSERVED_NO_TIME), Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing}, Optimization.ReInitCache{ComponentArrays.ComponentVector{Float64, Vector{Float64}, Tuple{ComponentArrays.Axis{(depvar = ViewAxis(1:217, Axis(layer_1 = ViewAxis(1:64, Axis(weight = ViewAxis(1:56, ShapedAxis((8, 7), NamedTuple())), bias = ViewAxis(57:64, ShapedAxis((8, 1), NamedTuple())))), layer_2 = ViewAxis(65:136, Axis(weight = ViewAxis(1:64, ShapedAxis((8, 8), NamedTuple())), bias = ViewAxis(65:72, ShapedAxis((8, 1), NamedTuple())))), layer_3 = ViewAxis(137:208, Axis(weight = ViewAxis(1:64, ShapedAxis((8, 8), NamedTuple())), bias = ViewAxis(65:72, ShapedAxis((8, 1), NamedTuple())))), layer_4 = ViewAxis(209:217, Axis(weight = ViewAxis(1:8, ShapedAxis((1, 8), NamedTuple())), bias = ViewAxis(9:9, ShapedAxis((1, 1), NamedTuple())))))), p = 218:220)}}}, SciMLBase.NullParameters}}}, Optimization.var"#231#240"{Optimization.var"#227#236"{OptimizationFunction{true, Optimization.AutoZygote, NeuralPDE.var"#full_loss_function#308"{NeuralPDE.var"#null_nonadaptive_loss#115", Vector{NeuralPDE.var"#74#75"{NeuralPDE.var"#209#210"{RuntimeGeneratedFunctions.RuntimeGeneratedFunction{(:cord, Symbol("##θ#293"), :phi, :derivative, :integral, :u, :p), NeuralPDE.var"#_RGF_ModTag", NeuralPDE.var"#_RGF_ModTag", (0xad10bb64, 0x1d2657c2, 0xc9c67c1a, 0xbaf4020a, 0xc4a1861b), Expr}, NeuralPDE.var"#12#13", NeuralPDE.var"#269#276"{NeuralPDE.var"#269#270#277"{typeof(NeuralPDE.numeric_derivative)}, Dict{Symbol, Int64}, Dict{Symbol, Int64}, GridTraining{Float64}}, typeof(NeuralPDE.numeric_derivative), NeuralPDE.Phi{Lux.Chain{NamedTuple{(:layer_1, :layer_2, :layer_3, :layer_4), Tuple{Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(identity), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}}}}, NamedTuple{(:layer_1, :layer_2, :layer_3, :layer_4), NTuple{4, NamedTuple{(), Tuple{}}}}}, Vector{Float64}}, Matrix{Float64}}}, Vector{NeuralPDE.var"#74#75"{NeuralPDE.var"#209#210"{RuntimeGeneratedFunctions.RuntimeGeneratedFunction{(:cord, Symbol("##θ#293"), :phi, :derivative, :integral, :u, :p), NeuralPDE.var"#_RGF_ModTag", NeuralPDE.var"#_RGF_ModTag", (0x7cf59a19, 0x0162d753, 0x260a99e5, 0xc4bc113e, 0x070b6155), Expr}, NeuralPDE.var"#12#13", NeuralPDE.var"#269#276"{NeuralPDE.var"#269#270#277"{typeof(NeuralPDE.numeric_derivative)}, Dict{Symbol, Int64}, Dict{Symbol, Int64}, GridTraining{Float64}}, typeof(NeuralPDE.numeric_derivative), NeuralPDE.Phi{Lux.Chain{NamedTuple{(:layer_1, :layer_2, :layer_3, :layer_4), Tuple{Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(identity), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}}}}, NamedTuple{(:layer_1, :layer_2, :layer_3, :layer_4), NTuple{4, NamedTuple{(), Tuple{}}}}}, Vector{Float64}}, Matrix{Float64}}}, NeuralPDE.PINNRepresentation, Bool, Vector{Int64}, Int64, NeuralPDE.Phi{Lux.Chain{NamedTuple{(:layer_1, :layer_2, :layer_3, :layer_4), Tuple{Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(identity), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}}}}, NamedTuple{(:layer_1, :layer_2, :layer_3, :layer_4), NTuple{4, NamedTuple{(), Tuple{}}}}}, typeof(additional_loss), Bool, Vector{Float64}}, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, typeof(SciMLBase.DEFAULT_OBSERVED_NO_TIME), Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing}, Optimization.ReInitCache{ComponentArrays.ComponentVector{Float64, Vector{Float64}, Tuple{ComponentArrays.Axis{(depvar = ViewAxis(1:217, Axis(layer_1 = ViewAxis(1:64, Axis(weight = ViewAxis(1:56, ShapedAxis((8, 7), NamedTuple())), bias = ViewAxis(57:64, ShapedAxis((8, 1), NamedTuple())))), layer_2 = ViewAxis(65:136, Axis(weight = ViewAxis(1:64, ShapedAxis((8, 8), NamedTuple())), bias = ViewAxis(65:72, ShapedAxis((8, 1), NamedTuple())))), layer_3 = ViewAxis(137:208, Axis(weight = ViewAxis(1:64, ShapedAxis((8, 8), NamedTuple())), bias = ViewAxis(65:72, ShapedAxis((8, 1), NamedTuple())))), layer_4 = ViewAxis(209:217, Axis(weight = ViewAxis(1:8, ShapedAxis((1, 8), NamedTuple())), bias = ViewAxis(9:9, ShapedAxis((1, 1), NamedTuple())))))), p = 218:220)}}}, SciMLBase.NullParameters}}}, Optimization.var"#235#244", Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, typeof(SciMLBase.DEFAULT_OBSERVED_NO_TIME), Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing}, Optimization.ReInitCache{ComponentArrays.ComponentVector{Float64, Vector{Float64}, Tuple{ComponentArrays.Axis{(depvar = ViewAxis(1:217, Axis(layer_1 = ViewAxis(1:64, Axis(weight = ViewAxis(1:56, ShapedAxis((8, 7), NamedTuple())), bias = ViewAxis(57:64, ShapedAxis((8, 1), NamedTuple())))), layer_2 = ViewAxis(65:136, Axis(weight = ViewAxis(1:64, ShapedAxis((8, 8), NamedTuple())), bias = ViewAxis(65:72, ShapedAxis((8, 1), NamedTuple())))), layer_3 = ViewAxis(137:208, Axis(weight = ViewAxis(1:64, ShapedAxis((8, 8), NamedTuple())), bias = ViewAxis(65:72, ShapedAxis((8, 1), NamedTuple())))), layer_4 = ViewAxis(209:217, Axis(weight = ViewAxis(1:8, ShapedAxis((1, 8), NamedTuple())), bias = ViewAxis(9:9, ShapedAxis((1, 1), NamedTuple())))))), p = 218:220)}}}, SciMLBase.NullParameters}, Nothing, Nothing, Nothing, Nothing, Nothing, BFGS{LineSearches.InitialStatic{Float64}, LineSearches.HagerZhang{Float64, Base.RefValue{Bool}}, Nothing, Nothing, Flat}, Base.Iterators.Cycle{Tuple{Optimization.NullData}}, Bool, var"#3#4"}}})
    @ Optim C:\Users\loren\.julia\packages\Optim\V8ZEC\src\multivariate\optimize\optimize.jl:36
 [58] __solve(cache::OptimizationOptimJL.OptimJLOptimizationCache{OptimizationFunction{false, Optimization.AutoZygote, OptimizationFunction{true, Optimization.AutoZygote, NeuralPDE.var"#full_loss_function#308"{NeuralPDE.var"#null_nonadaptive_loss#115", Vector{NeuralPDE.var"#74#75"{NeuralPDE.var"#209#210"{RuntimeGeneratedFunctions.RuntimeGeneratedFunction{(:cord, Symbol("##θ#293"), :phi, :derivative, :integral, :u, :p), NeuralPDE.var"#_RGF_ModTag", NeuralPDE.var"#_RGF_ModTag", (0xad10bb64, 0x1d2657c2, 0xc9c67c1a, 0xbaf4020a, 0xc4a1861b), Expr}, NeuralPDE.var"#12#13", NeuralPDE.var"#269#276"{NeuralPDE.var"#269#270#277"{typeof(NeuralPDE.numeric_derivative)}, Dict{Symbol, Int64}, Dict{Symbol, Int64}, GridTraining{Float64}}, typeof(NeuralPDE.numeric_derivative), NeuralPDE.Phi{Lux.Chain{NamedTuple{(:layer_1, :layer_2, :layer_3, :layer_4), Tuple{Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(identity), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}}}}, NamedTuple{(:layer_1, :layer_2, :layer_3, :layer_4), NTuple{4, NamedTuple{(), Tuple{}}}}}, Vector{Float64}}, Matrix{Float64}}}, Vector{NeuralPDE.var"#74#75"{NeuralPDE.var"#209#210"{RuntimeGeneratedFunctions.RuntimeGeneratedFunction{(:cord, Symbol("##θ#293"), :phi, :derivative, :integral, :u, :p), NeuralPDE.var"#_RGF_ModTag", NeuralPDE.var"#_RGF_ModTag", (0x7cf59a19, 0x0162d753, 0x260a99e5, 0xc4bc113e, 0x070b6155), Expr}, NeuralPDE.var"#12#13", NeuralPDE.var"#269#276"{NeuralPDE.var"#269#270#277"{typeof(NeuralPDE.numeric_derivative)}, Dict{Symbol, Int64}, Dict{Symbol, Int64}, GridTraining{Float64}}, typeof(NeuralPDE.numeric_derivative), NeuralPDE.Phi{Lux.Chain{NamedTuple{(:layer_1, :layer_2, :layer_3, :layer_4), Tuple{Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(identity), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}}}}, NamedTuple{(:layer_1, :layer_2, :layer_3, :layer_4), NTuple{4, NamedTuple{(), Tuple{}}}}}, Vector{Float64}}, Matrix{Float64}}}, NeuralPDE.PINNRepresentation, Bool, Vector{Int64}, Int64, NeuralPDE.Phi{Lux.Chain{NamedTuple{(:layer_1, :layer_2, :layer_3, :layer_4), Tuple{Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(identity), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}}}}, NamedTuple{(:layer_1, :layer_2, :layer_3, :layer_4), NTuple{4, NamedTuple{(), Tuple{}}}}}, typeof(additional_loss), Bool, Vector{Float64}}, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, typeof(SciMLBase.DEFAULT_OBSERVED_NO_TIME), Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing}, Optimization.var"#228#237"{Optimization.var"#227#236"{OptimizationFunction{true, Optimization.AutoZygote, NeuralPDE.var"#full_loss_function#308"{NeuralPDE.var"#null_nonadaptive_loss#115", Vector{NeuralPDE.var"#74#75"{NeuralPDE.var"#209#210"{RuntimeGeneratedFunctions.RuntimeGeneratedFunction{(:cord, Symbol("##θ#293"), :phi, :derivative, :integral, :u, :p), NeuralPDE.var"#_RGF_ModTag", NeuralPDE.var"#_RGF_ModTag", (0xad10bb64, 0x1d2657c2, 0xc9c67c1a, 0xbaf4020a, 0xc4a1861b), Expr}, NeuralPDE.var"#12#13", NeuralPDE.var"#269#276"{NeuralPDE.var"#269#270#277"{typeof(NeuralPDE.numeric_derivative)}, Dict{Symbol, Int64}, Dict{Symbol, Int64}, GridTraining{Float64}}, typeof(NeuralPDE.numeric_derivative), NeuralPDE.Phi{Lux.Chain{NamedTuple{(:layer_1, :layer_2, :layer_3, :layer_4), Tuple{Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(identity), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}}}}, NamedTuple{(:layer_1, :layer_2, :layer_3, :layer_4), NTuple{4, NamedTuple{(), Tuple{}}}}}, Vector{Float64}}, Matrix{Float64}}}, Vector{NeuralPDE.var"#74#75"{NeuralPDE.var"#209#210"{RuntimeGeneratedFunctions.RuntimeGeneratedFunction{(:cord, Symbol("##θ#293"), :phi, :derivative, :integral, :u, :p), NeuralPDE.var"#_RGF_ModTag", NeuralPDE.var"#_RGF_ModTag", (0x7cf59a19, 0x0162d753, 0x260a99e5, 0xc4bc113e, 0x070b6155), Expr}, NeuralPDE.var"#12#13", NeuralPDE.var"#269#276"{NeuralPDE.var"#269#270#277"{typeof(NeuralPDE.numeric_derivative)}, Dict{Symbol, Int64}, Dict{Symbol, Int64}, GridTraining{Float64}}, typeof(NeuralPDE.numeric_derivative), NeuralPDE.Phi{Lux.Chain{NamedTuple{(:layer_1, :layer_2, :layer_3, :layer_4), Tuple{Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(identity), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}}}}, NamedTuple{(:layer_1, :layer_2, :layer_3, :layer_4), NTuple{4, NamedTuple{(), Tuple{}}}}}, Vector{Float64}}, Matrix{Float64}}}, NeuralPDE.PINNRepresentation, Bool, Vector{Int64}, Int64, NeuralPDE.Phi{Lux.Chain{NamedTuple{(:layer_1, :layer_2, :layer_3, :layer_4), Tuple{Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(identity), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}}}}, NamedTuple{(:layer_1, :layer_2, :layer_3, :layer_4), NTuple{4, NamedTuple{(), Tuple{}}}}}, typeof(additional_loss), Bool, Vector{Float64}}, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, typeof(SciMLBase.DEFAULT_OBSERVED_NO_TIME), Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing}, Optimization.ReInitCache{ComponentArrays.ComponentVector{Float64, Vector{Float64}, Tuple{ComponentArrays.Axis{(depvar = ViewAxis(1:217, Axis(layer_1 = ViewAxis(1:64, Axis(weight = ViewAxis(1:56, ShapedAxis((8, 7), NamedTuple())), bias = ViewAxis(57:64, ShapedAxis((8, 1), NamedTuple())))), layer_2 = ViewAxis(65:136, Axis(weight = ViewAxis(1:64, ShapedAxis((8, 8), NamedTuple())), bias = ViewAxis(65:72, ShapedAxis((8, 1), NamedTuple())))), layer_3 = ViewAxis(137:208, Axis(weight = ViewAxis(1:64, ShapedAxis((8, 8), NamedTuple())), bias = ViewAxis(65:72, ShapedAxis((8, 1), NamedTuple())))), layer_4 = ViewAxis(209:217, Axis(weight = ViewAxis(1:8, ShapedAxis((1, 8), NamedTuple())), bias = ViewAxis(9:9, ShapedAxis((1, 1), NamedTuple())))))), p = 218:220)}}}, SciMLBase.NullParameters}}}, Optimization.var"#231#240"{Optimization.var"#227#236"{OptimizationFunction{true, Optimization.AutoZygote, NeuralPDE.var"#full_loss_function#308"{NeuralPDE.var"#null_nonadaptive_loss#115", Vector{NeuralPDE.var"#74#75"{NeuralPDE.var"#209#210"{RuntimeGeneratedFunctions.RuntimeGeneratedFunction{(:cord, Symbol("##θ#293"), :phi, :derivative, :integral, :u, :p), NeuralPDE.var"#_RGF_ModTag", NeuralPDE.var"#_RGF_ModTag", (0xad10bb64, 0x1d2657c2, 0xc9c67c1a, 0xbaf4020a, 0xc4a1861b), Expr}, NeuralPDE.var"#12#13", NeuralPDE.var"#269#276"{NeuralPDE.var"#269#270#277"{typeof(NeuralPDE.numeric_derivative)}, Dict{Symbol, Int64}, Dict{Symbol, Int64}, GridTraining{Float64}}, typeof(NeuralPDE.numeric_derivative), NeuralPDE.Phi{Lux.Chain{NamedTuple{(:layer_1, :layer_2, :layer_3, :layer_4), Tuple{Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(identity), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}}}}, NamedTuple{(:layer_1, :layer_2, :layer_3, :layer_4), NTuple{4, NamedTuple{(), Tuple{}}}}}, Vector{Float64}}, Matrix{Float64}}}, Vector{NeuralPDE.var"#74#75"{NeuralPDE.var"#209#210"{RuntimeGeneratedFunctions.RuntimeGeneratedFunction{(:cord, Symbol("##θ#293"), :phi, :derivative, :integral, :u, :p), NeuralPDE.var"#_RGF_ModTag", NeuralPDE.var"#_RGF_ModTag", (0x7cf59a19, 0x0162d753, 0x260a99e5, 0xc4bc113e, 0x070b6155), Expr}, NeuralPDE.var"#12#13", NeuralPDE.var"#269#276"{NeuralPDE.var"#269#270#277"{typeof(NeuralPDE.numeric_derivative)}, Dict{Symbol, Int64}, Dict{Symbol, Int64}, GridTraining{Float64}}, typeof(NeuralPDE.numeric_derivative), NeuralPDE.Phi{Lux.Chain{NamedTuple{(:layer_1, :layer_2, :layer_3, :layer_4), Tuple{Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(identity), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}}}}, NamedTuple{(:layer_1, :layer_2, :layer_3, :layer_4), NTuple{4, NamedTuple{(), Tuple{}}}}}, Vector{Float64}}, Matrix{Float64}}}, NeuralPDE.PINNRepresentation, Bool, Vector{Int64}, Int64, NeuralPDE.Phi{Lux.Chain{NamedTuple{(:layer_1, :layer_2, :layer_3, :layer_4), Tuple{Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(identity), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}}}}, NamedTuple{(:layer_1, :layer_2, :layer_3, :layer_4), NTuple{4, NamedTuple{(), Tuple{}}}}}, typeof(additional_loss), Bool, Vector{Float64}}, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, typeof(SciMLBase.DEFAULT_OBSERVED_NO_TIME), Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing}, Optimization.ReInitCache{ComponentArrays.ComponentVector{Float64, Vector{Float64}, Tuple{ComponentArrays.Axis{(depvar = ViewAxis(1:217, Axis(layer_1 = ViewAxis(1:64, Axis(weight = ViewAxis(1:56, ShapedAxis((8, 7), NamedTuple())), bias = ViewAxis(57:64, ShapedAxis((8, 1), NamedTuple())))), layer_2 = ViewAxis(65:136, Axis(weight = ViewAxis(1:64, ShapedAxis((8, 8), NamedTuple())), bias = ViewAxis(65:72, ShapedAxis((8, 1), NamedTuple())))), layer_3 = ViewAxis(137:208, Axis(weight = ViewAxis(1:64, ShapedAxis((8, 8), NamedTuple())), bias = ViewAxis(65:72, ShapedAxis((8, 1), NamedTuple())))), layer_4 = ViewAxis(209:217, Axis(weight = ViewAxis(1:8, ShapedAxis((1, 8), NamedTuple())), bias = ViewAxis(9:9, ShapedAxis((1, 1), NamedTuple())))))), p = 218:220)}}}, SciMLBase.NullParameters}}}, Optimization.var"#235#244", Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, typeof(SciMLBase.DEFAULT_OBSERVED_NO_TIME), Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing}, Optimization.ReInitCache{ComponentArrays.ComponentVector{Float64, Vector{Float64}, Tuple{ComponentArrays.Axis{(depvar = ViewAxis(1:217, Axis(layer_1 = ViewAxis(1:64, Axis(weight = ViewAxis(1:56, ShapedAxis((8, 7), NamedTuple())), bias = ViewAxis(57:64, ShapedAxis((8, 1), NamedTuple())))), layer_2 = ViewAxis(65:136, Axis(weight = ViewAxis(1:64, ShapedAxis((8, 8), NamedTuple())), bias = ViewAxis(65:72, ShapedAxis((8, 1), NamedTuple())))), layer_3 = ViewAxis(137:208, Axis(weight = ViewAxis(1:64, ShapedAxis((8, 8), NamedTuple())), bias = ViewAxis(65:72, ShapedAxis((8, 1), NamedTuple())))), layer_4 = ViewAxis(209:217, Axis(weight = ViewAxis(1:8, ShapedAxis((1, 8), NamedTuple())), bias = ViewAxis(9:9, ShapedAxis((1, 1), NamedTuple())))))), p = 218:220)}}}, SciMLBase.NullParameters}, Nothing, Nothing, Nothing, Nothing, Nothing, BFGS{LineSearches.InitialStatic{Float64}, LineSearches.HagerZhang{Float64, Base.RefValue{Bool}}, Nothing, Nothing, Flat}, Base.Iterators.Cycle{Tuple{Optimization.NullData}}, Bool, var"#3#4"})
    @ OptimizationOptimJL C:\Users\loren\.julia\packages\OptimizationOptimJL\uRfW9\src\OptimizationOptimJL.jl:249
 [59] solve!(cache::OptimizationOptimJL.OptimJLOptimizationCache{OptimizationFunction{false, Optimization.AutoZygote, OptimizationFunction{true, Optimization.AutoZygote, NeuralPDE.var"#full_loss_function#308"{NeuralPDE.var"#null_nonadaptive_loss#115", Vector{NeuralPDE.var"#74#75"{NeuralPDE.var"#209#210"{RuntimeGeneratedFunctions.RuntimeGeneratedFunction{(:cord, Symbol("##θ#293"), :phi, :derivative, :integral, :u, :p), NeuralPDE.var"#_RGF_ModTag", NeuralPDE.var"#_RGF_ModTag", (0xad10bb64, 0x1d2657c2, 0xc9c67c1a, 0xbaf4020a, 0xc4a1861b), Expr}, NeuralPDE.var"#12#13", NeuralPDE.var"#269#276"{NeuralPDE.var"#269#270#277"{typeof(NeuralPDE.numeric_derivative)}, Dict{Symbol, Int64}, Dict{Symbol, Int64}, GridTraining{Float64}}, typeof(NeuralPDE.numeric_derivative), NeuralPDE.Phi{Lux.Chain{NamedTuple{(:layer_1, :layer_2, :layer_3, :layer_4), Tuple{Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(identity), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}}}}, NamedTuple{(:layer_1, :layer_2, :layer_3, :layer_4), NTuple{4, NamedTuple{(), Tuple{}}}}}, Vector{Float64}}, Matrix{Float64}}}, Vector{NeuralPDE.var"#74#75"{NeuralPDE.var"#209#210"{RuntimeGeneratedFunctions.RuntimeGeneratedFunction{(:cord, Symbol("##θ#293"), :phi, :derivative, :integral, :u, :p), NeuralPDE.var"#_RGF_ModTag", NeuralPDE.var"#_RGF_ModTag", (0x7cf59a19, 0x0162d753, 0x260a99e5, 0xc4bc113e, 0x070b6155), Expr}, NeuralPDE.var"#12#13", NeuralPDE.var"#269#276"{NeuralPDE.var"#269#270#277"{typeof(NeuralPDE.numeric_derivative)}, Dict{Symbol, Int64}, Dict{Symbol, Int64}, GridTraining{Float64}}, typeof(NeuralPDE.numeric_derivative), NeuralPDE.Phi{Lux.Chain{NamedTuple{(:layer_1, :layer_2, :layer_3, :layer_4), Tuple{Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(identity), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}}}}, NamedTuple{(:layer_1, :layer_2, :layer_3, :layer_4), NTuple{4, NamedTuple{(), Tuple{}}}}}, Vector{Float64}}, Matrix{Float64}}}, NeuralPDE.PINNRepresentation, Bool, Vector{Int64}, Int64, NeuralPDE.Phi{Lux.Chain{NamedTuple{(:layer_1, :layer_2, :layer_3, :layer_4), Tuple{Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(identity), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}}}}, NamedTuple{(:layer_1, :layer_2, :layer_3, :layer_4), NTuple{4, NamedTuple{(), Tuple{}}}}}, typeof(additional_loss), Bool, Vector{Float64}}, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, typeof(SciMLBase.DEFAULT_OBSERVED_NO_TIME), Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing}, Optimization.var"#228#237"{Optimization.var"#227#236"{OptimizationFunction{true, Optimization.AutoZygote, NeuralPDE.var"#full_loss_function#308"{NeuralPDE.var"#null_nonadaptive_loss#115", Vector{NeuralPDE.var"#74#75"{NeuralPDE.var"#209#210"{RuntimeGeneratedFunctions.RuntimeGeneratedFunction{(:cord, Symbol("##θ#293"), :phi, :derivative, :integral, :u, :p), NeuralPDE.var"#_RGF_ModTag", NeuralPDE.var"#_RGF_ModTag", (0xad10bb64, 0x1d2657c2, 0xc9c67c1a, 0xbaf4020a, 0xc4a1861b), Expr}, NeuralPDE.var"#12#13", NeuralPDE.var"#269#276"{NeuralPDE.var"#269#270#277"{typeof(NeuralPDE.numeric_derivative)}, Dict{Symbol, Int64}, Dict{Symbol, Int64}, GridTraining{Float64}}, typeof(NeuralPDE.numeric_derivative), NeuralPDE.Phi{Lux.Chain{NamedTuple{(:layer_1, :layer_2, :layer_3, :layer_4), Tuple{Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(identity), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}}}}, NamedTuple{(:layer_1, :layer_2, :layer_3, :layer_4), NTuple{4, NamedTuple{(), Tuple{}}}}}, Vector{Float64}}, Matrix{Float64}}}, Vector{NeuralPDE.var"#74#75"{NeuralPDE.var"#209#210"{RuntimeGeneratedFunctions.RuntimeGeneratedFunction{(:cord, Symbol("##θ#293"), :phi, :derivative, :integral, :u, :p), NeuralPDE.var"#_RGF_ModTag", NeuralPDE.var"#_RGF_ModTag", (0x7cf59a19, 0x0162d753, 0x260a99e5, 0xc4bc113e, 0x070b6155), Expr}, NeuralPDE.var"#12#13", NeuralPDE.var"#269#276"{NeuralPDE.var"#269#270#277"{typeof(NeuralPDE.numeric_derivative)}, Dict{Symbol, Int64}, Dict{Symbol, Int64}, GridTraining{Float64}}, typeof(NeuralPDE.numeric_derivative), NeuralPDE.Phi{Lux.Chain{NamedTuple{(:layer_1, :layer_2, :layer_3, :layer_4), Tuple{Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(identity), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}}}}, NamedTuple{(:layer_1, :layer_2, :layer_3, :layer_4), NTuple{4, NamedTuple{(), Tuple{}}}}}, Vector{Float64}}, Matrix{Float64}}}, NeuralPDE.PINNRepresentation, Bool, Vector{Int64}, Int64, NeuralPDE.Phi{Lux.Chain{NamedTuple{(:layer_1, :layer_2, :layer_3, :layer_4), Tuple{Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(identity), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}}}}, NamedTuple{(:layer_1, :layer_2, :layer_3, :layer_4), NTuple{4, NamedTuple{(), Tuple{}}}}}, typeof(additional_loss), Bool, Vector{Float64}}, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, typeof(SciMLBase.DEFAULT_OBSERVED_NO_TIME), Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing}, Optimization.ReInitCache{ComponentArrays.ComponentVector{Float64, Vector{Float64}, Tuple{ComponentArrays.Axis{(depvar = ViewAxis(1:217, Axis(layer_1 = ViewAxis(1:64, Axis(weight = ViewAxis(1:56, ShapedAxis((8, 7), NamedTuple())), bias = ViewAxis(57:64, ShapedAxis((8, 1), NamedTuple())))), layer_2 = ViewAxis(65:136, Axis(weight = ViewAxis(1:64, ShapedAxis((8, 8), NamedTuple())), bias = ViewAxis(65:72, ShapedAxis((8, 1), NamedTuple())))), layer_3 = ViewAxis(137:208, Axis(weight = ViewAxis(1:64, ShapedAxis((8, 8), NamedTuple())), bias = ViewAxis(65:72, ShapedAxis((8, 1), NamedTuple())))), layer_4 = ViewAxis(209:217, Axis(weight = ViewAxis(1:8, ShapedAxis((1, 8), NamedTuple())), bias = ViewAxis(9:9, ShapedAxis((1, 1), NamedTuple())))))), p = 218:220)}}}, SciMLBase.NullParameters}}}, Optimization.var"#231#240"{Optimization.var"#227#236"{OptimizationFunction{true, Optimization.AutoZygote, NeuralPDE.var"#full_loss_function#308"{NeuralPDE.var"#null_nonadaptive_loss#115", Vector{NeuralPDE.var"#74#75"{NeuralPDE.var"#209#210"{RuntimeGeneratedFunctions.RuntimeGeneratedFunction{(:cord, Symbol("##θ#293"), :phi, :derivative, :integral, :u, :p), NeuralPDE.var"#_RGF_ModTag", NeuralPDE.var"#_RGF_ModTag", (0xad10bb64, 0x1d2657c2, 0xc9c67c1a, 0xbaf4020a, 0xc4a1861b), Expr}, NeuralPDE.var"#12#13", NeuralPDE.var"#269#276"{NeuralPDE.var"#269#270#277"{typeof(NeuralPDE.numeric_derivative)}, Dict{Symbol, Int64}, Dict{Symbol, Int64}, GridTraining{Float64}}, typeof(NeuralPDE.numeric_derivative), NeuralPDE.Phi{Lux.Chain{NamedTuple{(:layer_1, :layer_2, :layer_3, :layer_4), Tuple{Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(identity), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}}}}, NamedTuple{(:layer_1, :layer_2, :layer_3, :layer_4), NTuple{4, NamedTuple{(), Tuple{}}}}}, Vector{Float64}}, Matrix{Float64}}}, Vector{NeuralPDE.var"#74#75"{NeuralPDE.var"#209#210"{RuntimeGeneratedFunctions.RuntimeGeneratedFunction{(:cord, Symbol("##θ#293"), :phi, :derivative, :integral, :u, :p), NeuralPDE.var"#_RGF_ModTag", NeuralPDE.var"#_RGF_ModTag", (0x7cf59a19, 0x0162d753, 0x260a99e5, 0xc4bc113e, 0x070b6155), Expr}, NeuralPDE.var"#12#13", NeuralPDE.var"#269#276"{NeuralPDE.var"#269#270#277"{typeof(NeuralPDE.numeric_derivative)}, Dict{Symbol, Int64}, Dict{Symbol, Int64}, GridTraining{Float64}}, typeof(NeuralPDE.numeric_derivative), NeuralPDE.Phi{Lux.Chain{NamedTuple{(:layer_1, :layer_2, :layer_3, :layer_4), Tuple{Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(identity), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}}}}, NamedTuple{(:layer_1, :layer_2, :layer_3, :layer_4), NTuple{4, NamedTuple{(), Tuple{}}}}}, Vector{Float64}}, Matrix{Float64}}}, NeuralPDE.PINNRepresentation, Bool, Vector{Int64}, Int64, NeuralPDE.Phi{Lux.Chain{NamedTuple{(:layer_1, :layer_2, :layer_3, :layer_4), Tuple{Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(identity), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}}}}, NamedTuple{(:layer_1, :layer_2, :layer_3, :layer_4), NTuple{4, NamedTuple{(), Tuple{}}}}}, typeof(additional_loss), Bool, Vector{Float64}}, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, typeof(SciMLBase.DEFAULT_OBSERVED_NO_TIME), Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing}, Optimization.ReInitCache{ComponentArrays.ComponentVector{Float64, Vector{Float64}, Tuple{ComponentArrays.Axis{(depvar = ViewAxis(1:217, Axis(layer_1 = ViewAxis(1:64, Axis(weight = ViewAxis(1:56, ShapedAxis((8, 7), NamedTuple())), bias = ViewAxis(57:64, ShapedAxis((8, 1), NamedTuple())))), layer_2 = ViewAxis(65:136, Axis(weight = ViewAxis(1:64, ShapedAxis((8, 8), NamedTuple())), bias = ViewAxis(65:72, ShapedAxis((8, 1), NamedTuple())))), layer_3 = ViewAxis(137:208, Axis(weight = ViewAxis(1:64, ShapedAxis((8, 8), NamedTuple())), bias = ViewAxis(65:72, ShapedAxis((8, 1), NamedTuple())))), layer_4 = ViewAxis(209:217, Axis(weight = ViewAxis(1:8, ShapedAxis((1, 8), NamedTuple())), bias = ViewAxis(9:9, ShapedAxis((1, 1), NamedTuple())))))), p = 218:220)}}}, SciMLBase.NullParameters}}}, Optimization.var"#235#244", Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, typeof(SciMLBase.DEFAULT_OBSERVED_NO_TIME), Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing}, Optimization.ReInitCache{ComponentArrays.ComponentVector{Float64, Vector{Float64}, Tuple{ComponentArrays.Axis{(depvar = ViewAxis(1:217, Axis(layer_1 = ViewAxis(1:64, Axis(weight = ViewAxis(1:56, ShapedAxis((8, 7), NamedTuple())), bias = ViewAxis(57:64, ShapedAxis((8, 1), NamedTuple())))), layer_2 = ViewAxis(65:136, Axis(weight = ViewAxis(1:64, ShapedAxis((8, 8), NamedTuple())), bias = ViewAxis(65:72, ShapedAxis((8, 1), NamedTuple())))), layer_3 = ViewAxis(137:208, Axis(weight = ViewAxis(1:64, ShapedAxis((8, 8), NamedTuple())), bias = ViewAxis(65:72, ShapedAxis((8, 1), NamedTuple())))), layer_4 = ViewAxis(209:217, Axis(weight = ViewAxis(1:8, ShapedAxis((1, 8), NamedTuple())), bias = ViewAxis(9:9, ShapedAxis((1, 1), NamedTuple())))))), p = 218:220)}}}, SciMLBase.NullParameters}, Nothing, Nothing, Nothing, Nothing, Nothing, BFGS{LineSearches.InitialStatic{Float64}, LineSearches.HagerZhang{Float64, Base.RefValue{Bool}}, Nothing, Nothing, Flat}, Base.Iterators.Cycle{Tuple{Optimization.NullData}}, Bool, var"#3#4"})
    @ SciMLBase C:\Users\loren\.julia\packages\SciMLBase\QqtZA\src\solve.jl:160
 [60] solve(::OptimizationProblem{true, OptimizationFunction{true, Optimization.AutoZygote, NeuralPDE.var"#full_loss_function#308"{NeuralPDE.var"#null_nonadaptive_loss#115", Vector{NeuralPDE.var"#74#75"{NeuralPDE.var"#209#210"{RuntimeGeneratedFunctions.RuntimeGeneratedFunction{(:cord, Symbol("##θ#293"), :phi, :derivative, :integral, :u, :p), NeuralPDE.var"#_RGF_ModTag", NeuralPDE.var"#_RGF_ModTag", (0xad10bb64, 0x1d2657c2, 0xc9c67c1a, 0xbaf4020a, 0xc4a1861b), Expr}, NeuralPDE.var"#12#13", NeuralPDE.var"#269#276"{NeuralPDE.var"#269#270#277"{typeof(NeuralPDE.numeric_derivative)}, Dict{Symbol, Int64}, Dict{Symbol, Int64}, GridTraining{Float64}}, typeof(NeuralPDE.numeric_derivative), NeuralPDE.Phi{Lux.Chain{NamedTuple{(:layer_1, :layer_2, :layer_3, :layer_4), Tuple{Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(identity), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}}}}, NamedTuple{(:layer_1, :layer_2, :layer_3, :layer_4), NTuple{4, NamedTuple{(), Tuple{}}}}}, Vector{Float64}}, Matrix{Float64}}}, Vector{NeuralPDE.var"#74#75"{NeuralPDE.var"#209#210"{RuntimeGeneratedFunctions.RuntimeGeneratedFunction{(:cord, Symbol("##θ#293"), :phi, :derivative, :integral, :u, :p), NeuralPDE.var"#_RGF_ModTag", NeuralPDE.var"#_RGF_ModTag", (0x7cf59a19, 0x0162d753, 0x260a99e5, 0xc4bc113e, 0x070b6155), Expr}, NeuralPDE.var"#12#13", NeuralPDE.var"#269#276"{NeuralPDE.var"#269#270#277"{typeof(NeuralPDE.numeric_derivative)}, Dict{Symbol, Int64}, Dict{Symbol, Int64}, GridTraining{Float64}}, typeof(NeuralPDE.numeric_derivative), NeuralPDE.Phi{Lux.Chain{NamedTuple{(:layer_1, :layer_2, :layer_3, :layer_4), Tuple{Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(identity), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}}}}, NamedTuple{(:layer_1, :layer_2, :layer_3, :layer_4), NTuple{4, NamedTuple{(), Tuple{}}}}}, Vector{Float64}}, Matrix{Float64}}}, NeuralPDE.PINNRepresentation, Bool, Vector{Int64}, Int64, NeuralPDE.Phi{Lux.Chain{NamedTuple{(:layer_1, :layer_2, :layer_3, :layer_4), Tuple{Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(identity), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}}}}, NamedTuple{(:layer_1, :layer_2, :layer_3, :layer_4), NTuple{4, NamedTuple{(), Tuple{}}}}}, typeof(additional_loss), Bool, Vector{Float64}}, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, typeof(SciMLBase.DEFAULT_OBSERVED_NO_TIME), Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing}, ComponentArrays.ComponentVector{Float64, Vector{Float64}, Tuple{ComponentArrays.Axis{(depvar = ViewAxis(1:217, Axis(layer_1 = ViewAxis(1:64, Axis(weight = ViewAxis(1:56, ShapedAxis((8, 7), NamedTuple())), bias = ViewAxis(57:64, ShapedAxis((8, 1), NamedTuple())))), layer_2 = ViewAxis(65:136, Axis(weight = ViewAxis(1:64, ShapedAxis((8, 8), NamedTuple())), bias = ViewAxis(65:72, ShapedAxis((8, 1), NamedTuple())))), layer_3 = ViewAxis(137:208, Axis(weight = ViewAxis(1:64, ShapedAxis((8, 8), NamedTuple())), bias = ViewAxis(65:72, ShapedAxis((8, 1), NamedTuple())))), layer_4 = ViewAxis(209:217, Axis(weight = ViewAxis(1:8, ShapedAxis((1, 8), NamedTuple())), bias = ViewAxis(9:9, ShapedAxis((1, 1), NamedTuple())))))), p = 218:220)}}}, SciMLBase.NullParameters, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Base.Pairs{Symbol, Union{}, Tuple{}, NamedTuple{(), Tuple{}}}}, ::BFGS{LineSearches.InitialStatic{Float64}, LineSearches.HagerZhang{Float64, Base.RefValue{Bool}}, Nothing, Nothing, Flat}; kwargs::Base.Pairs{Symbol, Any, Tuple{Symbol, Symbol}, NamedTuple{(:maxiters, :callback), Tuple{Int64, var"#3#4"}}})
    @ SciMLBase C:\Users\loren\.julia\packages\SciMLBase\QqtZA\src\solve.jl:81

Astitva Aggarwal · Answer 13 · Fri Nov 10 2023 01:56:28 GMT+0800 (China Standard Time)

hey, sorry got busy with some other work. taking a look at this now. also while your at it why not join our slack? link , make sure to join the sciml-bridged channel.

Astitva Aggarwal · Answer 14 · Fri Nov 10 2023 02:55:07 GMT+0800 (China Standard Time)

hey @LorenzoBottaccioli im confused about your problem formulation, you have mentioned indvars=[t,T_ext,Q_heating,Q_cooling,Q_sun,Q_lights,Q_equipment]
depvars = [T_in] but in your equations every variable seems to be a function of t only, so wouldnt you make only t your independant variable?

Astitva Aggarwal · Answer 15 · Fri Nov 10 2023 21:21:01 GMT+0800 (China Standard Time)

@LorenzoBottaccioli this works for me, ive also reformulated the code as mentioned above.

using NeuralPDE, Lux, ModelingToolkit, DataFrames, CSV, DataLoaders, Flux, IntervalSets,
    Optimization, OptimizationOptimJL

# Definisci il modello dell'equazione differenziale
@parameters t, R, C, Cs

@variables T_in(..),
T_ext(..),
Q_heating(..),
Q_cooling(..),
Q_sun(..),
Q_lights(..),
Q_equipment(..)

# R, C, Cs = [1, 2, 3]

Dt = Differential(t)

# eqs = Dt(T_in(t)) ~ (-T_ext(t) + T_in(t)) / (R * C)
eqs = Dt(T_in(t)) ~ (T_ext(t) - T_in(t)) / (R * C) + Q_heating(t) / C - Q_cooling(t) / C +
                    Q_sun(t) / Cs + (Q_lights(t) + Q_equipment(t)) / C

domains = [t ∈ Interval(0.0, 365.0 * 24.0 * 60.0)]
bcs = [Dt(T_in(0.0)) ~ 4.48]

dt = 600.0  # 600 seconds (10 minute)

# Define the temporal space
tspan = (0.0, 365.0 * 24.0 * 60.0)  # Dati per un anno

# load sampled data from CSV
data = CSV.File("shoebox_free.csv") |> DataFrame

# Put the sampled data in dedicated variables
T_in_data = data."OSGB1000005735772_FLOOR_1:Zone Mean Air Temperature [C](TimeStep)"[1:500]
T_ext_data = data."Environment:Site Outdoor Air Drybulb Temperature [C](TimeStep)"[1:500]
Q_heating_data = data."OSGB1000005735772_FLOOR_1_HVAC:Zone Ideal Loads Zone Total Heating Rate [W](TimeStep)"[1:500]
Q_cooling_data = data."OSGB1000005735772_FLOOR_1_HVAC:Zone Ideal Loads Zone Total Cooling Rate [W](TimeStep)"[1:500]
Q_sun_data = data."Environment:Site Direct Solar Radiation Rate per Area [W/m2](TimeStep)"[1:500]
Q_lights_data = data."OSGB1000005735772_FLOOR_1:Zone Lights Total Heating Rate [W](TimeStep)"[1:500]
Q_equipment_data = data."OSGB1000005735772_FLOOR_1:Zone Electric Equipment Total Heating Rate [W](TimeStep)"[1:500]

t_data = collect(Float64, 1:size(data, 1))

dataloader = DataLoader([
    vcat(T_in_data,
        T_ext_data,
        Q_heating_data,
        Q_cooling_data,
        Q_sun_data,
        Q_lights_data,
        Q_equipment_data,
        t_data),
])

# Define the NN
input_dim = 1
hidden_units = 8
len = length(t_data)

chain1 = Flux.Chain(Flux.Dense(input_dim, hidden_units, σ), Flux.Dense(hidden_units, 1)) |>
         Flux.f64
chain2 = Flux.Chain(Flux.Dense(input_dim, hidden_units, σ), Flux.Dense(hidden_units, 1)) |>
         Flux.f64
chain3 = Flux.Chain(Flux.Dense(input_dim, hidden_units, σ), Flux.Dense(hidden_units, 1)) |>
         Flux.f64
chain4 = Flux.Chain(Flux.Dense(input_dim, hidden_units, σ), Flux.Dense(hidden_units, 1)) |>
         Flux.f64
chain5 = Flux.Chain(Flux.Dense(input_dim, hidden_units, σ), Flux.Dense(hidden_units, 1)) |>
         Flux.f64
chain6 = Flux.Chain(Flux.Dense(input_dim, hidden_units, σ), Flux.Dense(hidden_units, 1)) |>
         Flux.f64
chain7 = Flux.Chain(Flux.Dense(input_dim, hidden_units, σ), Flux.Dense(hidden_units, 1)) |>
         Flux.f64

#Define dependent and independent vatiables
indvars = [t]
depvars = [:T_in, :T_ext, :Q_heating, :Q_cooling, :Q_sun, :Q_lights, :Q_equipment]
u_ = hcat(T_in_data,
    T_ext_data,
    Q_heating_data,
    Q_cooling_data,
    Q_sun_data,
    Q_lights_data,
    Q_equipment_data)

# Define the loss(additional loss will be using all data vectors)
init_params = [Flux.destructure(c)[1]
               for c in [chain1, chain2, chain3, chain4, chain5, chain6, chain7]]
acum = [0; accumulate(+, length.(init_params))]
sep = [(acum[i] + 1):acum[i + 1] for i in 1:(length(acum) - 1)]

function additional_loss(phi, θ, p)
    return sum(sum(abs2, phi[i](t_data[1:500]', θ[sep[i]]) .- u_[:, [i]]') / 500.0
               for i in 1:1:7)
end

@named pde_system = NeuralPDE.PDESystem(eqs,
    bcs,
    domains,
    [t],
    [T_in(t), T_ext(t), Q_heating(t), Q_cooling(t), Q_sun(t), Q_lights(t), Q_equipment(t)
    ],
    [R, C, Cs],
    defaults = Dict([R => 1.0, C => 1.0, Cs => 1.0]))#[R, C, Cs])

discretization = NeuralPDE.PhysicsInformedNN([
        chain1,
        chain2,
        chain3,
        chain4,
        chain5,
        chain6, chain7,
    ],
    NeuralPDE.GridTraining(dt), param_estim = true, additional_loss = additional_loss)
prob = NeuralPDE.discretize(pde_system, discretization)

# Parameter Optimization
res = Optimization.solve(prob,
    BFGS(),
    maxiters = 1000,
    callback = callback)

LorenzoBottaccioli · Answer 16 · Sat Nov 11 2023 01:50:03 GMT+0800 (China Standard Time)

@AstitvaAggarwal tnx. There are few things I don't get:

Why there is the need of 7 chains? I need only an NN to predict the T_in variation not all the others T_ext(t),Q_heating(t),Q_cooling(t),Q_sun(t),Q_lights(t),Q_equipment(t) are exogenpous inputs that only vary with time.
What you have done in the additional loss?
When do I use the data in dataloader?

Astitva Aggarwal · Answer 17 · Mon Nov 13 2023 02:48:44 GMT+0800 (China Standard Time)

1> using chains you could train it not only at the timepoints in dataset but also on other timepoints in the domain of t. also you were taking the rest at dependant variables on t so learning that variable made sense to me.
2> additonal loss adds MSE loss for all 7 variables for its data and that particular NNs output.
3>No idea, i thought you were using that later on in your code so ignored that

LorenzoBottaccioli · Answer 18 · Mon Nov 13 2023 17:21:41 GMT+0800 (China Standard Time)

@AstitvaAggarwal I tried to explain the objective better. The goal is to predict T_in that varieties over time with the provided equation eqs. T_in is influenced by exogenous inputs that vary in time with some of the exogenous inputs that are controllable. So there is no need to predict any of those variables. The overall goal hence is to learn the unknown parameters of the eqs and use PINN to predict the variation of T_in in time influenced by the exogenous inputs.

I have tried to review the code you provided to work as I mentioned above but I get an error related to the dimension that is due to the presence of the unknown parameter.

using NeuralPDE, Lux, ModelingToolkit, DataFrames, CSV, DataLoaders, Flux, IntervalSets,
    Optimization, OptimizationOptimJL

# Definisci il modello dell'equazione differenziale
@parameters t, R, C, Cs

@variables T_in(..),
T_ext(..),
Q_heating(..),
Q_cooling(..),
Q_sun(..),
Q_lights(..),
Q_equipment(..)

# R, C, Cs = [1, 2, 3]

Dt = Differential(t)

# eqs = Dt(T_in(t)) ~ (-T_ext(t) + T_in(t)) / (R * C)
eqs = Dt(T_in(t)) ~ (T_in(t)-T_ext(t)) / (R * C) + Q_heating(t) / C - Q_cooling(t) / C +
                    Q_sun(t) / Cs + (Q_lights(t) + Q_equipment(t)) / C

domains = [t ∈ Interval(0.0, 365.0 * 24.0 * 60.0)]
bcs = [Dt(T_in(0.0)) ~ 4.48]

dt = 600.0  # 600 seconds (10 minute)

# Define the temporal space
tspan = (0.0, 365.0 * 24.0 * 60.0)  

# load sampled data from CSV
data = CSV.File("shoebox_free.csv") |> DataFrame

# Put the sampled data in dedicated variables
T_in_data = data."OSGB1000005735772_FLOOR_1:Zone Mean Air Temperature [C](TimeStep)"[1:500]
T_ext_data = data."Environment:Site Outdoor Air Drybulb Temperature [C](TimeStep)"[1:500]
Q_heating_data = data."OSGB1000005735772_FLOOR_1_HVAC:Zone Ideal Loads Zone Total Heating Rate [W](TimeStep)"[1:500]
Q_cooling_data = data."OSGB1000005735772_FLOOR_1_HVAC:Zone Ideal Loads Zone Total Cooling Rate [W](TimeStep)"[1:500]
Q_sun_data = data."Environment:Site Direct Solar Radiation Rate per Area [W/m2](TimeStep)"[1:500]
Q_lights_data = data."OSGB1000005735772_FLOOR_1:Zone Lights Total Heating Rate [W](TimeStep)"[1:500]
Q_equipment_data = data."OSGB1000005735772_FLOOR_1:Zone Electric Equipment Total Heating Rate [W](TimeStep)"[1:500]

t_data = collect(Float64, 1:size(data, 1))

dataloader = DataLoader([
    vcat(T_in_data,
        T_ext_data,
        Q_heating_data,
        Q_cooling_data,
        Q_sun_data,
        Q_lights_data,
        Q_equipment_data,
        t_data),
])

# Define the NN
input_dim = 7
hidden_units = 8
len = length(t_data)

chain1 = Flux.Chain(Flux.Dense(input_dim, hidden_units, σ),Flux.Dense(hidden_units, hidden_units, σ), Flux.Dense(hidden_units, 1)) |>
         Flux.f64

indvars = [t]
depvars = [:T_in, :T_ext, :Q_heating, :Q_cooling, :Q_sun, :Q_lights, :Q_equipment]
u_ = hcat(T_in_data,
    T_ext_data,
    Q_heating_data,
    Q_cooling_data,
    Q_sun_data,
    Q_lights_data,
    Q_equipment_data)

init_params = [Flux.destructure(chain1)[1]]
acum = [0; accumulate(+, length.(init_params))]
sep = [(acum[i] + 1):acum[i + 1] for i in 1:(length(acum) - 1)]

function additional_loss(phi, θ, p)
    return sum(sum(abs2, phi(t_data, θ[sep[1]]) .- u_[:,1]))
end

discretization = NeuralPDE.PhysicsInformedNN(chain1,
    NeuralPDE.GridTraining(dt), param_estim = true, additional_loss = additional_loss)
prob = NeuralPDE.discretize(pde_system, discretization)

# Parameter Optimization
res = Optimization.solve(prob,
    BFGS(),
    maxiters = 1000)

Then I got

 DimensionMismatch: Rebuild expected a vector of length 145, got 147

Stacktrace:
  [1] #_rebuild#48
    @ C:\Users\loren\.julia\packages\Optimisers\er7zv\src\destructure.jl:95 [inlined]
  [2] _rebuild
    @ C:\Users\loren\.julia\packages\Optimisers\er7zv\src\destructure.jl:94 [inlined]
  [3] #rrule#53
    @ C:\Users\loren\.julia\packages\Optimisers\er7zv\src\destructure.jl:136 [inlined]
  [4] rrule
    @ C:\Users\loren\.julia\packages\Optimisers\er7zv\src\destructure.jl:134 [inlined]
  [5] rrule
    @ C:\Users\loren\.julia\packages\ChainRulesCore\7MWx2\src\rules.jl:134 [inlined]
  [6] chain_rrule
    @ C:\Users\loren\.julia\packages\Zygote\YYT6v\src\compiler\chainrules.jl:223 [inlined]
  [7] macro expansion
    @ C:\Users\loren\.julia\packages\Zygote\YYT6v\src\compiler\interface2.jl:101 [inlined]
  [8] _pullback
    @ C:\Users\loren\.julia\packages\Zygote\YYT6v\src\compiler\interface2.jl:101 [inlined]
  [9] _pullback
    @ C:\Users\loren\.julia\packages\Optimisers\er7zv\src\destructure.jl:59 [inlined]
 [10] _pullback
    @ C:\Users\loren\.julia\packages\NeuralPDE\F4RlZ\src\pinn_types.jl:371 [inlined]
 [11] _pullback(::Zygote.Context{false}, ::NeuralPDE.Phi{Optimisers.Restructure{Flux.Chain{Tuple{Flux.Dense{typeof(σ), Matrix{Float64}, Vector{Float64}}, Flux.Dense{typeof(σ), Matrix{Float64}, Vector{Float64}}, Flux.Dense{typeof(identity), Matrix{Float64}, Vector{Float64}}}}, NamedTuple{(:layers,), Tuple{Tuple{NamedTuple{(:weight, :bias, :σ), Tuple{Int64, Int64, Tuple{}}}, NamedTuple{(:weight, :bias, :σ), Tuple{Int64, Int64, Tuple{}}}, NamedTuple{(:weight, :bias, :σ), Tuple{Int64, Int64, Tuple{}}}}}}}, Nothing}, ::Matrix{Float64}, ::Vector{Float64})
    @ Zygote C:\Users\loren\.julia\packages\Zygote\YYT6v\src\compiler\interface2.jl:0
 [12] _pullback
    @ C:\Users\loren\.julia\packages\NeuralPDE\F4RlZ\src\pinn_types.jl:375 [inlined]
 [13] _pullback(::Zygote.Context{false}, ::NeuralPDE.var"#12#13", ::Matrix{Float64}, ::Vector{Float64}, ::NeuralPDE.Phi{Optimisers.Restructure{Flux.Chain{Tuple{Flux.Dense{typeof(σ), Matrix{Float64}, Vector{Float64}}, Flux.Dense{typeof(σ), Matrix{Float64}, Vector{Float64}}, Flux.Dense{typeof(identity), Matrix{Float64}, Vector{Float64}}}}, NamedTuple{(:layers,), Tuple{Tuple{NamedTuple{(:weight, :bias, :σ), Tuple{Int64, Int64, Tuple{}}}, NamedTuple{(:weight, :bias, :σ), Tuple{Int64, Int64, Tuple{}}}, NamedTuple{(:weight, :bias, :σ), Tuple{Int64, Int64, Tuple{}}}}}}}, Nothing})
    @ Zygote C:\Users\loren\.julia\packages\Zygote\YYT6v\src\compiler\interface2.jl:0
 [14] _pullback
    @ C:\Users\loren\.julia\packages\NeuralPDE\F4RlZ\src\pinn_types.jl:410 [inlined]
 [15] _pullback(::Zygote.Context{false}, ::typeof(NeuralPDE.numeric_derivative), ::NeuralPDE.Phi{Optimisers.Restructure{Flux.Chain{Tuple{Flux.Dense{typeof(σ), Matrix{Float64}, Vector{Float64}}, Flux.Dense{typeof(σ), Matrix{Float64}, Vector{Float64}}, Flux.Dense{typeof(identity), Matrix{Float64}, Vector{Float64}}}}, NamedTuple{(:layers,), Tuple{Tuple{NamedTuple{(:weight, :bias, :σ), Tuple{Int64, Int64, Tuple{}}}, NamedTuple{(:weight, :bias, :σ), Tuple{Int64, Int64, Tuple{}}}, NamedTuple{(:weight, :bias, :σ), Tuple{Int64, Int64, Tuple{}}}}}}}, Nothing}, ::NeuralPDE.var"#12#13", ::Matrix{Float64}, ::Vector{Vector{Float64}}, ::Int64, ::Vector{Float64})
    @ Zygote C:\Users\loren\.julia\packages\Zygote\YYT6v\src\compiler\interface2.jl:0
 [16] macro expansion
    @ C:\Users\loren\.julia\packages\NeuralPDE\F4RlZ\src\discretize.jl:169 [inlined]
 [17] macro expansion
    @ C:\Users\loren\.julia\packages\RuntimeGeneratedFunctions\Yo8zx\src\RuntimeGeneratedFunctions.jl:163 [inlined]
 [18] macro expansion
    @ .\none:0 [inlined]
 [19] _pullback
    @ .\none:0 [inlined]
 [20] _pullback(::Zygote.Context{false}, ::typeof(RuntimeGeneratedFunctions.generated_callfunc), ::RuntimeGeneratedFunctions.RuntimeGeneratedFunction{(:cord, Symbol("##θ#293"), :phi, :derivative, :integral, :u, :p), NeuralPDE.var"#_RGF_ModTag", NeuralPDE.var"#_RGF_ModTag", (0x64128365, 0xe99c13c8, 0x3553718d, 0xc14f6af0, 0x258a1b45), Expr}, ::Matrix{Float64}, ::Vector{Float64}, ::NeuralPDE.Phi{Optimisers.Restructure{Flux.Chain{Tuple{Flux.Dense{typeof(σ), Matrix{Float64}, Vector{Float64}}, Flux.Dense{typeof(σ), Matrix{Float64}, Vector{Float64}}, Flux.Dense{typeof(identity), Matrix{Float64}, Vector{Float64}}}}, NamedTuple{(:layers,), Tuple{Tuple{NamedTuple{(:weight, :bias, :σ), Tuple{Int64, Int64, Tuple{}}}, NamedTuple{(:weight, :bias, :σ), Tuple{Int64, Int64, Tuple{}}}, NamedTuple{(:weight, :bias, :σ), Tuple{Int64, Int64, Tuple{}}}}}}}, Nothing}, ::typeof(NeuralPDE.numeric_derivative), ::NeuralPDE.var"#269#276"{NeuralPDE.var"#269#270#277"{typeof(NeuralPDE.numeric_derivative)}, Dict{Symbol, Int64}, Dict{Symbol, Int64}, GridTraining{Float64}}, ::NeuralPDE.var"#12#13", ::Vector{Float64})
    @ Zygote C:\Users\loren\.julia\packages\Zygote\YYT6v\src\compiler\interface2.jl:0
 [21] _apply(::Function, ::Vararg{Any})
    @ Core .\boot.jl:838
 [22] adjoint
    @ C:\Users\loren\.julia\packages\Zygote\YYT6v\src\lib\lib.jl:203 [inlined]
 [23] _pullback
    @ C:\Users\loren\.julia\packages\ZygoteRules\4nXuu\src\adjoint.jl:66 [inlined]
 [24] _pullback
    @ C:\Users\loren\.julia\packages\RuntimeGeneratedFunctions\Yo8zx\src\RuntimeGeneratedFunctions.jl:150 [inlined]
 [25] _pullback(::Zygote.Context{false}, ::RuntimeGeneratedFunctions.RuntimeGeneratedFunction{(:cord, Symbol("##θ#293"), :phi, :derivative, :integral, :u, :p), NeuralPDE.var"#_RGF_ModTag", NeuralPDE.var"#_RGF_ModTag", (0x64128365, 0xe99c13c8, 0x3553718d, 0xc14f6af0, 0x258a1b45), Expr}, ::Matrix{Float64}, ::Vector{Float64}, ::NeuralPDE.Phi{Optimisers.Restructure{Flux.Chain{Tuple{Flux.Dense{typeof(σ), Matrix{Float64}, Vector{Float64}}, Flux.Dense{typeof(σ), Matrix{Float64}, Vector{Float64}}, Flux.Dense{typeof(identity), Matrix{Float64}, Vector{Float64}}}}, NamedTuple{(:layers,), Tuple{Tuple{NamedTuple{(:weight, :bias, :σ), Tuple{Int64, Int64, Tuple{}}}, NamedTuple{(:weight, :bias, :σ), Tuple{Int64, Int64, Tuple{}}}, NamedTuple{(:weight, :bias, :σ), Tuple{Int64, Int64, Tuple{}}}}}}}, Nothing}, ::typeof(NeuralPDE.numeric_derivative), ::NeuralPDE.var"#269#276"{NeuralPDE.var"#269#270#277"{typeof(NeuralPDE.numeric_derivative)}, Dict{Symbol, Int64}, Dict{Symbol, Int64}, GridTraining{Float64}}, ::NeuralPDE.var"#12#13", ::Vector{Float64})
    @ Zygote C:\Users\loren\.julia\packages\Zygote\YYT6v\src\compiler\interface2.jl:0
 [26] _pullback
    @ C:\Users\loren\.julia\packages\NeuralPDE\F4RlZ\src\discretize.jl:192 [inlined]
 [27] _pullback(::Zygote.Context{false}, ::NeuralPDE.var"#209#210"{RuntimeGeneratedFunctions.RuntimeGeneratedFunction{(:cord, Symbol("##θ#293"), :phi, :derivative, :integral, :u, :p), NeuralPDE.var"#_RGF_ModTag", NeuralPDE.var"#_RGF_ModTag", (0x64128365, 0xe99c13c8, 0x3553718d, 0xc14f6af0, 0x258a1b45), Expr}, NeuralPDE.var"#12#13", NeuralPDE.var"#269#276"{NeuralPDE.var"#269#270#277"{typeof(NeuralPDE.numeric_derivative)}, Dict{Symbol, Int64}, Dict{Symbol, Int64}, GridTraining{Float64}}, typeof(NeuralPDE.numeric_derivative), NeuralPDE.Phi{Optimisers.Restructure{Flux.Chain{Tuple{Flux.Dense{typeof(σ), Matrix{Float64}, Vector{Float64}}, Flux.Dense{typeof(σ), Matrix{Float64}, Vector{Float64}}, Flux.Dense{typeof(identity), Matrix{Float64}, Vector{Float64}}}}, NamedTuple{(:layers,), Tuple{Tuple{NamedTuple{(:weight, :bias, :σ), Tuple{Int64, Int64, Tuple{}}}, NamedTuple{(:weight, :bias, :σ), Tuple{Int64, Int64, Tuple{}}}, NamedTuple{(:weight, :bias, :σ), Tuple{Int64, Int64, Tuple{}}}}}}}, Nothing}, Vector{Float64}}, ::Matrix{Float64}, ::Vector{Float64})
    @ Zygote C:\Users\loren\.julia\packages\Zygote\YYT6v\src\compiler\interface2.jl:0
 [28] _pullback
    @ C:\Users\loren\.julia\packages\NeuralPDE\F4RlZ\src\training_strategies.jl:48 [inlined]
 [29] _pullback(ctx::Zygote.Context{false}, f::NeuralPDE.var"#74#75"{NeuralPDE.var"#209#210"{RuntimeGeneratedFunctions.RuntimeGeneratedFunction{(:cord, Symbol("##θ#293"), :phi, :derivative, :integral, :u, :p), NeuralPDE.var"#_RGF_ModTag", NeuralPDE.var"#_RGF_ModTag", (0x64128365, 0xe99c13c8, 0x3553718d, 0xc14f6af0, 0x258a1b45), Expr}, NeuralPDE.var"#12#13", NeuralPDE.var"#269#276"{NeuralPDE.var"#269#270#277"{typeof(NeuralPDE.numeric_derivative)}, Dict{Symbol, Int64}, Dict{Symbol, Int64}, GridTraining{Float64}}, typeof(NeuralPDE.numeric_derivative), NeuralPDE.Phi{Optimisers.Restructure{Flux.Chain{Tuple{Flux.Dense{typeof(σ), Matrix{Float64}, Vector{Float64}}, Flux.Dense{typeof(σ), Matrix{Float64}, Vector{Float64}}, Flux.Dense{typeof(identity), Matrix{Float64}, Vector{Float64}}}}, NamedTuple{(:layers,), Tuple{Tuple{NamedTuple{(:weight, :bias, :σ), Tuple{Int64, Int64, Tuple{}}}, NamedTuple{(:weight, :bias, :σ), Tuple{Int64, Int64, Tuple{}}}, NamedTuple{(:weight, :bias, :σ), Tuple{Int64, Int64, Tuple{}}}}}}}, Nothing}, Vector{Float64}}, Matrix{Float64}}, args::Vector{Float64})
    @ Zygote C:\Users\loren\.julia\packages\Zygote\YYT6v\src\compiler\interface2.jl:0
 [30] _pullback
    @ .\none:0 [inlined]
 [31] _pullback(ctx::Zygote.Context{false}, f::NeuralPDE.var"#294#309"{Vector{Float64}}, args::NeuralPDE.var"#74#75"{NeuralPDE.var"#209#210"{RuntimeGeneratedFunctions.RuntimeGeneratedFunction{(:cord, Symbol("##θ#293"), :phi, :derivative, :integral, :u, :p), NeuralPDE.var"#_RGF_ModTag", NeuralPDE.var"#_RGF_ModTag", (0x64128365, 0xe99c13c8, 0x3553718d, 0xc14f6af0, 0x258a1b45), Expr}, NeuralPDE.var"#12#13", NeuralPDE.var"#269#276"{NeuralPDE.var"#269#270#277"{typeof(NeuralPDE.numeric_derivative)}, Dict{Symbol, Int64}, Dict{Symbol, Int64}, GridTraining{Float64}}, typeof(NeuralPDE.numeric_derivative), NeuralPDE.Phi{Optimisers.Restructure{Flux.Chain{Tuple{Flux.Dense{typeof(σ), Matrix{Float64}, Vector{Float64}}, Flux.Dense{typeof(σ), Matrix{Float64}, Vector{Float64}}, Flux.Dense{typeof(identity), Matrix{Float64}, Vector{Float64}}}}, NamedTuple{(:layers,), Tuple{Tuple{NamedTuple{(:weight, :bias, :σ), Tuple{Int64, Int64, Tuple{}}}, NamedTuple{(:weight, :bias, :σ), Tuple{Int64, Int64, Tuple{}}}, NamedTuple{(:weight, :bias, :σ), Tuple{Int64, Int64, Tuple{}}}}}}}, Nothing}, Vector{Float64}}, Matrix{Float64}})
    @ Zygote C:\Users\loren\.julia\packages\Zygote\YYT6v\src\compiler\interface2.jl:0
 [32] #661
    @ C:\Users\loren\.julia\packages\Zygote\YYT6v\src\lib\array.jl:183 [inlined]
 [33] iterate
    @ .\generator.jl:47 [inlined]
 [34] _collect(c::Vector{NeuralPDE.var"#74#75"{NeuralPDE.var"#209#210"{RuntimeGeneratedFunctions.RuntimeGeneratedFunction{(:cord, Symbol("##θ#293"), :phi, :derivative, :integral, :u, :p), NeuralPDE.var"#_RGF_ModTag", NeuralPDE.var"#_RGF_ModTag", (0x64128365, 0xe99c13c8, 0x3553718d, 0xc14f6af0, 0x258a1b45), Expr}, NeuralPDE.var"#12#13", NeuralPDE.var"#269#276"{NeuralPDE.var"#269#270#277"{typeof(NeuralPDE.numeric_derivative)}, Dict{Symbol, Int64}, Dict{Symbol, Int64}, GridTraining{Float64}}, typeof(NeuralPDE.numeric_derivative), NeuralPDE.Phi{Optimisers.Restructure{Flux.Chain{Tuple{Flux.Dense{typeof(σ), Matrix{Float64}, Vector{Float64}}, Flux.Dense{typeof(σ), Matrix{Float64}, Vector{Float64}}, Flux.Dense{typeof(identity), Matrix{Float64}, Vector{Float64}}}}, NamedTuple{(:layers,), Tuple{Tuple{NamedTuple{(:weight, :bias, :σ), Tuple{Int64, Int64, Tuple{}}}, NamedTuple{(:weight, :bias, :σ), Tuple{Int64, Int64, Tuple{}}}, NamedTuple{(:weight, :bias, :σ), Tuple{Int64, Int64, Tuple{}}}}}}}, Nothing}, Vector{Float64}}, Matrix{Float64}}}, itr::Base.Generator{Vector{NeuralPDE.var"#74#75"{NeuralPDE.var"#209#210"{RuntimeGeneratedFunctions.RuntimeGeneratedFunction{(:cord, Symbol("##θ#293"), :phi, :derivative, :integral, :u, :p), NeuralPDE.var"#_RGF_ModTag", NeuralPDE.var"#_RGF_ModTag", (0x64128365, 0xe99c13c8, 0x3553718d, 0xc14f6af0, 0x258a1b45), Expr}, NeuralPDE.var"#12#13", NeuralPDE.var"#269#276"{NeuralPDE.var"#269#270#277"{typeof(NeuralPDE.numeric_derivative)}, Dict{Symbol, Int64}, Dict{Symbol, Int64}, GridTraining{Float64}}, typeof(NeuralPDE.numeric_derivative), NeuralPDE.Phi{Optimisers.Restructure{Flux.Chain{Tuple{Flux.Dense{typeof(σ), Matrix{Float64}, Vector{Float64}}, Flux.Dense{typeof(σ), Matrix{Float64}, Vector{Float64}}, Flux.Dense{typeof(identity), Matrix{Float64}, Vector{Float64}}}}, NamedTuple{(:layers,), Tuple{Tuple{NamedTuple{(:weight, :bias, :σ), Tuple{Int64, Int64, Tuple{}}}, NamedTuple{(:weight, :bias, :σ), Tuple{Int64, Int64, Tuple{}}}, NamedTuple{(:weight, :bias, :σ), Tuple{Int64, Int64, Tuple{}}}}}}}, Nothing}, Vector{Float64}}, Matrix{Float64}}}, Zygote.var"#661#665"{Zygote.Context{false}, NeuralPDE.var"#294#309"{Vector{Float64}}}}, #unused#::Base.EltypeUnknown, isz::Base.HasShape{1})
    @ Base .\array.jl:802
 [35] collect_similar
    @ .\array.jl:711 [inlined]
 [36] map
    @ .\abstractarray.jl:3261 [inlined]
 [37] ∇map(cx::Zygote.Context{false}, f::NeuralPDE.var"#294#309"{Vector{Float64}}, args::Vector{NeuralPDE.var"#74#75"{NeuralPDE.var"#209#210"{RuntimeGeneratedFunctions.RuntimeGeneratedFunction{(:cord, Symbol("##θ#293"), :phi, :derivative, :integral, :u, :p), NeuralPDE.var"#_RGF_ModTag", NeuralPDE.var"#_RGF_ModTag", (0x64128365, 0xe99c13c8, 0x3553718d, 0xc14f6af0, 0x258a1b45), Expr}, NeuralPDE.var"#12#13", NeuralPDE.var"#269#276"{NeuralPDE.var"#269#270#277"{typeof(NeuralPDE.numeric_derivative)}, Dict{Symbol, Int64}, Dict{Symbol, Int64}, GridTraining{Float64}}, typeof(NeuralPDE.numeric_derivative), NeuralPDE.Phi{Optimisers.Restructure{Flux.Chain{Tuple{Flux.Dense{typeof(σ), Matrix{Float64}, Vector{Float64}}, Flux.Dense{typeof(σ), Matrix{Float64}, Vector{Float64}}, Flux.Dense{typeof(identity), Matrix{Float64}, Vector{Float64}}}}, NamedTuple{(:layers,), Tuple{Tuple{NamedTuple{(:weight, :bias, :σ), Tuple{Int64, Int64, Tuple{}}}, NamedTuple{(:weight, :bias, :σ), Tuple{Int64, Int64, Tuple{}}}, NamedTuple{(:weight, :bias, :σ), Tuple{Int64, Int64, Tuple{}}}}}}}, Nothing}, Vector{Float64}}, Matrix{Float64}}})
    @ Zygote C:\Users\loren\.julia\packages\Zygote\YYT6v\src\lib\array.jl:183
 [38] _pullback
    @ C:\Users\loren\.julia\packages\Zygote\YYT6v\src\lib\array.jl:226 [inlined]
 [39] _pullback
    @ C:\Users\loren\.julia\packages\NeuralPDE\F4RlZ\src\discretize.jl:592 [inlined]
 [40] _pullback(::Zygote.Context{false}, ::NeuralPDE.var"#full_loss_function#308"{NeuralPDE.var"#null_nonadaptive_loss#115", Vector{NeuralPDE.var"#74#75"{NeuralPDE.var"#209#210"{RuntimeGeneratedFunctions.RuntimeGeneratedFunction{(:cord, Symbol("##θ#293"), :phi, :derivative, :integral, :u, :p), NeuralPDE.var"#_RGF_ModTag", NeuralPDE.var"#_RGF_ModTag", (0x03d798f9, 0x34fa0c63, 0xe7b60143, 0x8b2612da, 0x6b36183f), Expr}, NeuralPDE.var"#12#13", NeuralPDE.var"#269#276"{NeuralPDE.var"#269#270#277"{typeof(NeuralPDE.numeric_derivative)}, Dict{Symbol, Int64}, Dict{Symbol, Int64}, GridTraining{Float64}}, typeof(NeuralPDE.numeric_derivative), NeuralPDE.Phi{Optimisers.Restructure{Flux.Chain{Tuple{Flux.Dense{typeof(σ), Matrix{Float64}, Vector{Float64}}, Flux.Dense{typeof(σ), Matrix{Float64}, Vector{Float64}}, Flux.Dense{typeof(identity), Matrix{Float64}, Vector{Float64}}}}, NamedTuple{(:layers,), Tuple{Tuple{NamedTuple{(:weight, :bias, :σ), Tuple{Int64, Int64, Tuple{}}}, NamedTuple{(:weight, :bias, :σ), Tuple{Int64, Int64, Tuple{}}}, NamedTuple{(:weight, :bias, :σ), Tuple{Int64, Int64, Tuple{}}}}}}}, Nothing}, Vector{Float64}}, Matrix{Float64}}}, Vector{NeuralPDE.var"#74#75"{NeuralPDE.var"#209#210"{RuntimeGeneratedFunctions.RuntimeGeneratedFunction{(:cord, Symbol("##θ#293"), :phi, :derivative, :integral, :u, :p), NeuralPDE.var"#_RGF_ModTag", NeuralPDE.var"#_RGF_ModTag", (0x64128365, 0xe99c13c8, 0x3553718d, 0xc14f6af0, 0x258a1b45), Expr}, NeuralPDE.var"#12#13", NeuralPDE.var"#269#276"{NeuralPDE.var"#269#270#277"{typeof(NeuralPDE.numeric_derivative)}, Dict{Symbol, Int64}, Dict{Symbol, Int64}, GridTraining{Float64}}, typeof(NeuralPDE.numeric_derivative), NeuralPDE.Phi{Optimisers.Restructure{Flux.Chain{Tuple{Flux.Dense{typeof(σ), Matrix{Float64}, Vector{Float64}}, Flux.Dense{typeof(σ), Matrix{Float64}, Vector{Float64}}, Flux.Dense{typeof(identity), Matrix{Float64}, Vector{Float64}}}}, NamedTuple{(:layers,), Tuple{Tuple{NamedTuple{(:weight, :bias, :σ), Tuple{Int64, Int64, Tuple{}}}, NamedTuple{(:weight, :bias, :σ), Tuple{Int64, Int64, Tuple{}}}, NamedTuple{(:weight, :bias, :σ), Tuple{Int64, Int64, Tuple{}}}}}}}, Nothing}, Vector{Float64}}, Matrix{Float64}}}, NeuralPDE.PINNRepresentation, Bool, Vector{Int64}, Int64, NeuralPDE.Phi{Optimisers.Restructure{Flux.Chain{Tuple{Flux.Dense{typeof(σ), Matrix{Float64}, Vector{Float64}}, Flux.Dense{typeof(σ), Matrix{Float64}, Vector{Float64}}, Flux.Dense{typeof(identity), Matrix{Float64}, Vector{Float64}}}}, NamedTuple{(:layers,), Tuple{Tuple{NamedTuple{(:weight, :bias, :σ), Tuple{Int64, Int64, Tuple{}}}, NamedTuple{(:weight, :bias, :σ), Tuple{Int64, Int64, Tuple{}}}, NamedTuple{(:weight, :bias, :σ), Tuple{Int64, Int64, Tuple{}}}}}}}, Nothing}, typeof(additional_loss), Bool, Vector{Float64}}, ::Vector{Float64}, ::SciMLBase.NullParameters)
    @ Zygote C:\Users\loren\.julia\packages\Zygote\YYT6v\src\compiler\interface2.jl:0
 [41] _apply(::Function, ::Vararg{Any})
    @ Core .\boot.jl:838
 [42] adjoint
    @ C:\Users\loren\.julia\packages\Zygote\YYT6v\src\lib\lib.jl:203 [inlined]
 [43] _pullback
    @ C:\Users\loren\.julia\packages\ZygoteRules\4nXuu\src\adjoint.jl:66 [inlined]
 [44] _pullback
    @ C:\Users\loren\.julia\packages\SciMLBase\QqtZA\src\scimlfunctions.jl:3580 [inlined]
 [45] _pullback(::Zygote.Context{false}, ::OptimizationFunction{true, Optimization.AutoZygote, NeuralPDE.var"#full_loss_function#308"{NeuralPDE.var"#null_nonadaptive_loss#115", Vector{NeuralPDE.var"#74#75"{NeuralPDE.var"#209#210"{RuntimeGeneratedFunctions.RuntimeGeneratedFunction{(:cord, Symbol("##θ#293"), :phi, :derivative, :integral, :u, :p), NeuralPDE.var"#_RGF_ModTag", NeuralPDE.var"#_RGF_ModTag", (0x03d798f9, 0x34fa0c63, 0xe7b60143, 0x8b2612da, 0x6b36183f), Expr}, NeuralPDE.var"#12#13", NeuralPDE.var"#269#276"{NeuralPDE.var"#269#270#277"{typeof(NeuralPDE.numeric_derivative)}, Dict{Symbol, Int64}, Dict{Symbol, Int64}, GridTraining{Float64}}, typeof(NeuralPDE.numeric_derivative), NeuralPDE.Phi{Optimisers.Restructure{Flux.Chain{Tuple{Flux.Dense{typeof(σ), Matrix{Float64}, Vector{Float64}}, Flux.Dense{typeof(σ), Matrix{Float64}, Vector{Float64}}, Flux.Dense{typeof(identity), Matrix{Float64}, Vector{Float64}}}}, NamedTuple{(:layers,), Tuple{Tuple{NamedTuple{(:weight, :bias, :σ), Tuple{Int64, Int64, Tuple{}}}, NamedTuple{(:weight, :bias, :σ), Tuple{Int64, Int64, Tuple{}}}, NamedTuple{(:weight, :bias, :σ), Tuple{Int64, Int64, Tuple{}}}}}}}, Nothing}, Vector{Float64}}, Matrix{Float64}}}, Vector{NeuralPDE.var"#74#75"{NeuralPDE.var"#209#210"{RuntimeGeneratedFunctions.RuntimeGeneratedFunction{(:cord, Symbol("##θ#293"), :phi, :derivative, :integral, :u, :p), NeuralPDE.var"#_RGF_ModTag", NeuralPDE.var"#_RGF_ModTag", (0x64128365, 0xe99c13c8, 0x3553718d, 0xc14f6af0, 0x258a1b45), Expr}, NeuralPDE.var"#12#13", NeuralPDE.var"#269#276"{NeuralPDE.var"#269#270#277"{typeof(NeuralPDE.numeric_derivative)}, Dict{Symbol, Int64}, Dict{Symbol, Int64}, GridTraining{Float64}}, typeof(NeuralPDE.numeric_derivative), NeuralPDE.Phi{Optimisers.Restructure{Flux.Chain{Tuple{Flux.Dense{typeof(σ), Matrix{Float64}, Vector{Float64}}, Flux.Dense{typeof(σ), Matrix{Float64}, Vector{Float64}}, Flux.Dense{typeof(identity), Matrix{Float64}, Vector{Float64}}}}, NamedTuple{(:layers,), Tuple{Tuple{NamedTuple{(:weight, :bias, :σ), Tuple{Int64, Int64, Tuple{}}}, NamedTuple{(:weight, :bias, :σ), Tuple{Int64, Int64, Tuple{}}}, NamedTuple{(:weight, :bias, :σ), Tuple{Int64, Int64, Tuple{}}}}}}}, Nothing}, Vector{Float64}}, Matrix{Float64}}}, NeuralPDE.PINNRepresentation, Bool, Vector{Int64}, Int64, NeuralPDE.Phi{Optimisers.Restructure{Flux.Chain{Tuple{Flux.Dense{typeof(σ), Matrix{Float64}, Vector{Float64}}, Flux.Dense{typeof(σ), Matrix{Float64}, Vector{Float64}}, Flux.Dense{typeof(identity), Matrix{Float64}, Vector{Float64}}}}, NamedTuple{(:layers,), Tuple{Tuple{NamedTuple{(:weight, :bias, :σ), Tuple{Int64, Int64, Tuple{}}}, NamedTuple{(:weight, :bias, :σ), Tuple{Int64, Int64, Tuple{}}}, NamedTuple{(:weight, :bias, :σ), Tuple{Int64, Int64, Tuple{}}}}}}}, Nothing}, typeof(additional_loss), Bool, Vector{Float64}}, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, typeof(SciMLBase.DEFAULT_OBSERVED_NO_TIME), Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing}, ::Vector{Float64}, ::SciMLBase.NullParameters)
    @ Zygote C:\Users\loren\.julia\packages\Zygote\YYT6v\src\compiler\interface2.jl:0
 [46] _apply(::Function, ::Vararg{Any})
    @ Core .\boot.jl:838
 [47] adjoint
    @ C:\Users\loren\.julia\packages\Zygote\YYT6v\src\lib\lib.jl:203 [inlined]
 [48] _pullback
    @ C:\Users\loren\.julia\packages\ZygoteRules\4nXuu\src\adjoint.jl:66 [inlined]
 [49] _pullback
    @ C:\Users\loren\.julia\packages\Optimization\vFala\src\function\zygote.jl:72 [inlined]
 [50] _pullback(ctx::Zygote.Context{false}, f::Optimization.var"#227#236"{OptimizationFunction{true, Optimization.AutoZygote, NeuralPDE.var"#full_loss_function#308"{NeuralPDE.var"#null_nonadaptive_loss#115", Vector{NeuralPDE.var"#74#75"{NeuralPDE.var"#209#210"{RuntimeGeneratedFunctions.RuntimeGeneratedFunction{(:cord, Symbol("##θ#293"), :phi, :derivative, :integral, :u, :p), NeuralPDE.var"#_RGF_ModTag", NeuralPDE.var"#_RGF_ModTag", (0x03d798f9, 0x34fa0c63, 0xe7b60143, 0x8b2612da, 0x6b36183f), Expr}, NeuralPDE.var"#12#13", NeuralPDE.var"#269#276"{NeuralPDE.var"#269#270#277"{typeof(NeuralPDE.numeric_derivative)}, Dict{Symbol, Int64}, Dict{Symbol, Int64}, GridTraining{Float64}}, typeof(NeuralPDE.numeric_derivative), NeuralPDE.Phi{Optimisers.Restructure{Flux.Chain{Tuple{Flux.Dense{typeof(σ), Matrix{Float64}, Vector{Float64}}, Flux.Dense{typeof(σ), Matrix{Float64}, Vector{Float64}}, Flux.Dense{typeof(identity), Matrix{Float64}, Vector{Float64}}}}, NamedTuple{(:layers,), Tuple{Tuple{NamedTuple{(:weight, :bias, :σ), Tuple{Int64, Int64, Tuple{}}}, NamedTuple{(:weight, :bias, :σ), Tuple{Int64, Int64, Tuple{}}}, NamedTuple{(:weight, :bias, :σ), Tuple{Int64, Int64, Tuple{}}}}}}}, Nothing}, Vector{Float64}}, Matrix{Float64}}}, Vector{NeuralPDE.var"#74#75"{NeuralPDE.var"#209#210"{RuntimeGeneratedFunctions.RuntimeGeneratedFunction{(:cord, Symbol("##θ#293"), :phi, :derivative, :integral, :u, :p), NeuralPDE.var"#_RGF_ModTag", NeuralPDE.var"#_RGF_ModTag", (0x64128365, 0xe99c13c8, 0x3553718d, 0xc14f6af0, 0x258a1b45), Expr}, NeuralPDE.var"#12#13", NeuralPDE.var"#269#276"{NeuralPDE.var"#269#270#277"{typeof(NeuralPDE.numeric_derivative)}, Dict{Symbol, Int64}, Dict{Symbol, Int64}, GridTraining{Float64}}, typeof(NeuralPDE.numeric_derivative), NeuralPDE.Phi{Optimisers.Restructure{Flux.Chain{Tuple{Flux.Dense{typeof(σ), Matrix{Float64}, Vector{Float64}}, Flux.Dense{typeof(σ), Matrix{Float64}, Vector{Float64}}, Flux.Dense{typeof(identity), Matrix{Float64}, Vector{Float64}}}}, NamedTuple{(:layers,), Tuple{Tuple{NamedTuple{(:weight, :bias, :σ), Tuple{Int64, Int64, Tuple{}}}, NamedTuple{(:weight, :bias, :σ), Tuple{Int64, Int64, Tuple{}}}, NamedTuple{(:weight, :bias, :σ), Tuple{Int64, Int64, Tuple{}}}}}}}, Nothing}, Vector{Float64}}, Matrix{Float64}}}, NeuralPDE.PINNRepresentation, Bool, Vector{Int64}, Int64, NeuralPDE.Phi{Optimisers.Restructure{Flux.Chain{Tuple{Flux.Dense{typeof(σ), Matrix{Float64}, Vector{Float64}}, Flux.Dense{typeof(σ), Matrix{Float64}, Vector{Float64}}, Flux.Dense{typeof(identity), Matrix{Float64}, Vector{Float64}}}}, NamedTuple{(:layers,), Tuple{Tuple{NamedTuple{(:weight, :bias, :σ), Tuple{Int64, Int64, Tuple{}}}, NamedTuple{(:weight, :bias, :σ), Tuple{Int64, Int64, Tuple{}}}, NamedTuple{(:weight, :bias, :σ), Tuple{Int64, Int64, Tuple{}}}}}}}, Nothing}, typeof(additional_loss), Bool, Vector{Float64}}, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, typeof(SciMLBase.DEFAULT_OBSERVED_NO_TIME), Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing}, Optimization.ReInitCache{Vector{Float64}, SciMLBase.NullParameters}}, args::Vector{Float64})
    @ Zygote C:\Users\loren\.julia\packages\Zygote\YYT6v\src\compiler\interface2.jl:0
 [51] _apply(::Function, ::Vararg{Any})
    @ Core .\boot.jl:838
 [52] adjoint
    @ C:\Users\loren\.julia\packages\Zygote\YYT6v\src\lib\lib.jl:203 [inlined]
 [53] _pullback
    @ C:\Users\loren\.julia\packages\ZygoteRules\4nXuu\src\adjoint.jl:66 [inlined]
 [54] _pullback
    @ C:\Users\loren\.julia\packages\Optimization\vFala\src\function\zygote.jl:76 [inlined]
 [55] _pullback(ctx::Zygote.Context{false}, f::Optimization.var"#229#238"{Tuple{}, Optimization.var"#227#236"{OptimizationFunction{true, Optimization.AutoZygote, NeuralPDE.var"#full_loss_function#308"{NeuralPDE.var"#null_nonadaptive_loss#115", Vector{NeuralPDE.var"#74#75"{NeuralPDE.var"#209#210"{RuntimeGeneratedFunctions.RuntimeGeneratedFunction{(:cord, Symbol("##θ#293"), :phi, :derivative, :integral, :u, :p), NeuralPDE.var"#_RGF_ModTag", NeuralPDE.var"#_RGF_ModTag", (0x03d798f9, 0x34fa0c63, 0xe7b60143, 0x8b2612da, 0x6b36183f), Expr}, NeuralPDE.var"#12#13", NeuralPDE.var"#269#276"{NeuralPDE.var"#269#270#277"{typeof(NeuralPDE.numeric_derivative)}, Dict{Symbol, Int64}, Dict{Symbol, Int64}, GridTraining{Float64}}, typeof(NeuralPDE.numeric_derivative), NeuralPDE.Phi{Optimisers.Restructure{Flux.Chain{Tuple{Flux.Dense{typeof(σ), Matrix{Float64}, Vector{Float64}}, Flux.Dense{typeof(σ), Matrix{Float64}, Vector{Float64}}, Flux.Dense{typeof(identity), Matrix{Float64}, Vector{Float64}}}}, NamedTuple{(:layers,), Tuple{Tuple{NamedTuple{(:weight, :bias, :σ), Tuple{Int64, Int64, Tuple{}}}, NamedTuple{(:weight, :bias, :σ), Tuple{Int64, Int64, Tuple{}}}, NamedTuple{(:weight, :bias, :σ), Tuple{Int64, Int64, Tuple{}}}}}}}, Nothing}, Vector{Float64}}, Matrix{Float64}}}, Vector{NeuralPDE.var"#74#75"{NeuralPDE.var"#209#210"{RuntimeGeneratedFunctions.RuntimeGeneratedFunction{(:cord, Symbol("##θ#293"), :phi, :derivative, :integral, :u, :p), NeuralPDE.var"#_RGF_ModTag", NeuralPDE.var"#_RGF_ModTag", (0x64128365, 0xe99c13c8, 0x3553718d, 0xc14f6af0, 0x258a1b45), Expr}, NeuralPDE.var"#12#13", NeuralPDE.var"#269#276"{NeuralPDE.var"#269#270#277"{typeof(NeuralPDE.numeric_derivative)}, Dict{Symbol, Int64}, Dict{Symbol, Int64}, GridTraining{Float64}}, typeof(NeuralPDE.numeric_derivative), NeuralPDE.Phi{Optimisers.Restructure{Flux.Chain{Tuple{Flux.Dense{typeof(σ), Matrix{Float64}, Vector{Float64}}, Flux.Dense{typeof(σ), Matrix{Float64}, Vector{Float64}}, Flux.Dense{typeof(identity), Matrix{Float64}, Vector{Float64}}}}, NamedTuple{(:layers,), Tuple{Tuple{NamedTuple{(:weight, :bias, :σ), Tuple{Int64, Int64, Tuple{}}}, NamedTuple{(:weight, :bias, :σ), Tuple{Int64, Int64, Tuple{}}}, NamedTuple{(:weight, :bias, :σ), Tuple{Int64, Int64, Tuple{}}}}}}}, Nothing}, Vector{Float64}}, Matrix{Float64}}}, NeuralPDE.PINNRepresentation, Bool, Vector{Int64}, Int64, NeuralPDE.Phi{Optimisers.Restructure{Flux.Chain{Tuple{Flux.Dense{typeof(σ), Matrix{Float64}, Vector{Float64}}, Flux.Dense{typeof(σ), Matrix{Float64}, Vector{Float64}}, Flux.Dense{typeof(identity), Matrix{Float64}, Vector{Float64}}}}, NamedTuple{(:layers,), Tuple{Tuple{NamedTuple{(:weight, :bias, :σ), Tuple{Int64, Int64, Tuple{}}}, NamedTuple{(:weight, :bias, :σ), Tuple{Int64, Int64, Tuple{}}}, NamedTuple{(:weight, :bias, :σ), Tuple{Int64, Int64, Tuple{}}}}}}}, Nothing}, typeof(additional_loss), Bool, Vector{Float64}}, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, typeof(SciMLBase.DEFAULT_OBSERVED_NO_TIME), Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing}, Optimization.ReInitCache{Vector{Float64}, SciMLBase.NullParameters}}}, args::Vector{Float64})
    @ Zygote C:\Users\loren\.julia\packages\Zygote\YYT6v\src\compiler\interface2.jl:0
 [56] pullback(f::Function, cx::Zygote.Context{false}, args::Vector{Float64})
    @ Zygote C:\Users\loren\.julia\packages\Zygote\YYT6v\src\compiler\interface.jl:44
 [57] pullback
    @ C:\Users\loren\.julia\packages\Zygote\YYT6v\src\compiler\interface.jl:42 [inlined]
 [58] gradient(f::Function, args::Vector{Float64})
    @ Zygote C:\Users\loren\.julia\packages\Zygote\YYT6v\src\compiler\interface.jl:96
 [59] (::Optimization.var"#228#237"{Optimization.var"#227#236"{OptimizationFunction{true, Optimization.AutoZygote, NeuralPDE.var"#full_loss_function#308"{NeuralPDE.var"#null_nonadaptive_loss#115", Vector{NeuralPDE.var"#74#75"{NeuralPDE.var"#209#210"{RuntimeGeneratedFunctions.RuntimeGeneratedFunction{(:cord, Symbol("##θ#293"), :phi, :derivative, :integral, :u, :p), NeuralPDE.var"#_RGF_ModTag", NeuralPDE.var"#_RGF_ModTag", (0x03d798f9, 0x34fa0c63, 0xe7b60143, 0x8b2612da, 0x6b36183f), Expr}, NeuralPDE.var"#12#13", NeuralPDE.var"#269#276"{NeuralPDE.var"#269#270#277"{typeof(NeuralPDE.numeric_derivative)}, Dict{Symbol, Int64}, Dict{Symbol, Int64}, GridTraining{Float64}}, typeof(NeuralPDE.numeric_derivative), NeuralPDE.Phi{Optimisers.Restructure{Flux.Chain{Tuple{Flux.Dense{typeof(σ), Matrix{Float64}, Vector{Float64}}, Flux.Dense{typeof(σ), Matrix{Float64}, Vector{Float64}}, Flux.Dense{typeof(identity), Matrix{Float64}, Vector{Float64}}}}, NamedTuple{(:layers,), Tuple{Tuple{NamedTuple{(:weight, :bias, :σ), Tuple{Int64, Int64, Tuple{}}}, NamedTuple{(:weight, :bias, :σ), Tuple{Int64, Int64, Tuple{}}}, NamedTuple{(:weight, :bias, :σ), Tuple{Int64, Int64, Tuple{}}}}}}}, Nothing}, Vector{Float64}}, Matrix{Float64}}}, Vector{NeuralPDE.var"#74#75"{NeuralPDE.var"#209#210"{RuntimeGeneratedFunctions.RuntimeGeneratedFunction{(:cord, Symbol("##θ#293"), :phi, :derivative, :integral, :u, :p), NeuralPDE.var"#_RGF_ModTag", NeuralPDE.var"#_RGF_ModTag", (0x64128365, 0xe99c13c8, 0x3553718d, 0xc14f6af0, 0x258a1b45), Expr}, NeuralPDE.var"#12#13", NeuralPDE.var"#269#276"{NeuralPDE.var"#269#270#277"{typeof(NeuralPDE.numeric_derivative)}, Dict{Symbol, Int64}, Dict{Symbol, Int64}, GridTraining{Float64}}, typeof(NeuralPDE.numeric_derivative), NeuralPDE.Phi{Optimisers.Restructure{Flux.Chain{Tuple{Flux.Dense{typeof(σ), Matrix{Float64}, Vector{Float64}}, Flux.Dense{typeof(σ), Matrix{Float64}, Vector{Float64}}, Flux.Dense{typeof(identity), Matrix{Float64}, Vector{Float64}}}}, NamedTuple{(:layers,), Tuple{Tuple{NamedTuple{(:weight, :bias, :σ), Tuple{Int64, Int64, Tuple{}}}, NamedTuple{(:weight, :bias, :σ), Tuple{Int64, Int64, Tuple{}}}, NamedTuple{(:weight, :bias, :σ), Tuple{Int64, Int64, Tuple{}}}}}}}, Nothing}, Vector{Float64}}, Matrix{Float64}}}, NeuralPDE.PINNRepresentation, Bool, Vector{Int64}, Int64, NeuralPDE.Phi{Optimisers.Restructure{Flux.Chain{Tuple{Flux.Dense{typeof(σ), Matrix{Float64}, Vector{Float64}}, Flux.Dense{typeof(σ), Matrix{Float64}, Vector{Float64}}, Flux.Dense{typeof(identity), Matrix{Float64}, Vector{Float64}}}}, NamedTuple{(:layers,), Tuple{Tuple{NamedTuple{(:weight, :bias, :σ), Tuple{Int64, Int64, Tuple{}}}, NamedTuple{(:weight, :bias, :σ), Tuple{Int64, Int64, Tuple{}}}, NamedTuple{(:weight, :bias, :σ), Tuple{Int64, Int64, Tuple{}}}}}}}, Nothing}, typeof(additional_loss), Bool, Vector{Float64}}, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, typeof(SciMLBase.DEFAULT_OBSERVED_NO_TIME), Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing}, Optimization.ReInitCache{Vector{Float64}, SciMLBase.NullParameters}}})(::Vector{Float64}, ::Vector{Float64})
    @ Optimization C:\Users\loren\.julia\packages\Optimization\vFala\src\function\zygote.jl:74
 [60] (::OptimizationOptimJL.var"#10#16"{OptimizationOptimJL.OptimJLOptimizationCache{OptimizationFunction{false, Optimization.AutoZygote, OptimizationFunction{true, Optimization.AutoZygote, NeuralPDE.var"#full_loss_function#308"{NeuralPDE.var"#null_nonadaptive_loss#115", Vector{NeuralPDE.var"#74#75"{NeuralPDE.var"#209#210"{RuntimeGeneratedFunctions.RuntimeGeneratedFunction{(:cord, Symbol("##θ#293"), :phi, :derivative, :integral, :u, :p), NeuralPDE.var"#_RGF_ModTag", NeuralPDE.var"#_RGF_ModTag", (0x03d798f9, 0x34fa0c63, 0xe7b60143, 0x8b2612da, 0x6b36183f), Expr}, NeuralPDE.var"#12#13", NeuralPDE.var"#269#276"{NeuralPDE.var"#269#270#277"{typeof(NeuralPDE.numeric_derivative)}, Dict{Symbol, Int64}, Dict{Symbol, Int64}, GridTraining{Float64}}, typeof(NeuralPDE.numeric_derivative), NeuralPDE.Phi{Optimisers.Restructure{Flux.Chain{Tuple{Flux.Dense{typeof(σ), Matrix{Float64}, Vector{Float64}}, Flux.Dense{typeof(σ), Matrix{Float64}, Vector{Float64}}, Flux.Dense{typeof(identity), Matrix{Float64}, Vector{Float64}}}}, NamedTuple{(:layers,), Tuple{Tuple{NamedTuple{(:weight, :bias, :σ), Tuple{Int64, Int64, Tuple{}}}, NamedTuple{(:weight, :bias, :σ), Tuple{Int64, Int64, Tuple{}}}, NamedTuple{(:weight, :bias, :σ), Tuple{Int64, Int64, Tuple{}}}}}}}, Nothing}, Vector{Float64}}, Matrix{Float64}}}, Vector{NeuralPDE.var"#74#75"{NeuralPDE.var"#209#210"{RuntimeGeneratedFunctions.RuntimeGeneratedFunction{(:cord, Symbol("##θ#293"), :phi, :derivative, :integral, :u, :p), NeuralPDE.var"#_RGF_ModTag", NeuralPDE.var"#_RGF_ModTag", (0x64128365, 0xe99c13c8, 0x3553718d, 0xc14f6af0, 0x258a1b45), Expr}, NeuralPDE.var"#12#13", NeuralPDE.var"#269#276"{NeuralPDE.var"#269#270#277"{typeof(NeuralPDE.numeric_derivative)}, Dict{Symbol, Int64}, Dict{Symbol, Int64}, GridTraining{Float64}}, typeof(NeuralPDE.numeric_derivative), NeuralPDE.Phi{Optimisers.Restructure{Flux.Chain{Tuple{Flux.Dense{typeof(σ), Matrix{Float64}, Vector{Float64}}, Flux.Dense{typeof(σ), Matrix{Float64}, Vector{Float64}}, Flux.Dense{typeof(identity), Matrix{Float64}, Vector{Float64}}}}, NamedTuple{(:layers,), Tuple{Tuple{NamedTuple{(:weight, :bias, :σ), Tuple{Int64, Int64, Tuple{}}}, NamedTuple{(:weight, :bias, :σ), Tuple{Int64, Int64, Tuple{}}}, NamedTuple{(:weight, :bias, :σ), Tuple{Int64, Int64, Tuple{}}}}}}}, Nothing}, Vector{Float64}}, Matrix{Float64}}}, NeuralPDE.PINNRepresentation, Bool, Vector{Int64}, Int64, NeuralPDE.Phi{Optimisers.Restructure{Flux.Chain{Tuple{Flux.Dense{typeof(σ), Matrix{Float64}, Vector{Float64}}, Flux.Dense{typeof(σ), Matrix{Float64}, Vector{Float64}}, Flux.Dense{typeof(identity), Matrix{Float64}, Vector{Float64}}}}, NamedTuple{(:layers,), Tuple{Tuple{NamedTuple{(:weight, :bias, :σ), Tuple{Int64, Int64, Tuple{}}}, NamedTuple{(:weight, :bias, :σ), Tuple{Int64, Int64, Tuple{}}}, NamedTuple{(:weight, :bias, :σ), Tuple{Int64, Int64, Tuple{}}}}}}}, Nothing}, typeof(additional_loss), Bool, Vector{Float64}}, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, typeof(SciMLBase.DEFAULT_OBSERVED_NO_TIME), Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing}, Optimization.var"#228#237"{Optimization.var"#227#236"{OptimizationFunction{true, Optimization.AutoZygote, NeuralPDE.var"#full_loss_function#308"{NeuralPDE.var"#null_nonadaptive_loss#115", Vector{NeuralPDE.var"#74#75"{NeuralPDE.var"#209#210"{RuntimeGeneratedFunctions.RuntimeGeneratedFunction{(:cord, Symbol("##θ#293"), :phi, :derivative, :integral, :u, :p), NeuralPDE.var"#_RGF_ModTag", NeuralPDE.var"#_RGF_ModTag", (0x03d798f9, 0x34fa0c63, 0xe7b60143, 0x8b2612da, 0x6b36183f), Expr}, NeuralPDE.var"#12#13", NeuralPDE.var"#269#276"{NeuralPDE.var"#269#270#277"{typeof(NeuralPDE.numeric_derivative)}, Dict{Symbol, Int64}, Dict{Symbol, Int64}, GridTraining{Float64}}, typeof(NeuralPDE.numeric_derivative), NeuralPDE.Phi{Optimisers.Restructure{Flux.Chain{Tuple{Flux.Dense{typeof(σ), Matrix{Float64}, Vector{Float64}}, Flux.Dense{typeof(σ), Matrix{Float64}, Vector{Float64}}, Flux.Dense{typeof(identity), Matrix{Float64}, Vector{Float64}}}}, NamedTuple{(:layers,), Tuple{Tuple{NamedTuple{(:weight, :bias, :σ), Tuple{Int64, Int64, Tuple{}}}, NamedTuple{(:weight, :bias, :σ), Tuple{Int64, Int64, Tuple{}}}, NamedTuple{(:weight, :bias, :σ), Tuple{Int64, Int64, Tuple{}}}}}}}, Nothing}, Vector{Float64}}, Matrix{Float64}}}, Vector{NeuralPDE.var"#74#75"{NeuralPDE.var"#209#210"{RuntimeGeneratedFunctions.RuntimeGeneratedFunction{(:cord, Symbol("##θ#293"), :phi, :derivative, :integral, :u, :p), NeuralPDE.var"#_RGF_ModTag", NeuralPDE.var"#_RGF_ModTag", (0x64128365, 0xe99c13c8, 0x3553718d, 0xc14f6af0, 0x258a1b45), Expr}, NeuralPDE.var"#12#13", NeuralPDE.var"#269#276"{NeuralPDE.var"#269#270#277"{typeof(NeuralPDE.numeric_derivative)}, Dict{Symbol, Int64}, Dict{Symbol, Int64}, GridTraining{Float64}}, typeof(NeuralPDE.numeric_derivative), NeuralPDE.Phi{Optimisers.Restructure{Flux.Chain{Tuple{Flux.Dense{typeof(σ), Matrix{Float64}, Vector{Float64}}, Flux.Dense{typeof(σ), Matrix{Float64}, Vector{Float64}}, Flux.Dense{typeof(identity), Matrix{Float64}, Vector{Float64}}}}, NamedTuple{(:layers,), Tuple{Tuple{NamedTuple{(:weight, :bias, :σ), Tuple{Int64, Int64, Tuple{}}}, NamedTuple{(:weight, :bias, :σ), Tuple{Int64, Int64, Tuple{}}}, NamedTuple{(:weight, :bias, :σ), Tuple{Int64, Int64, Tuple{}}}}}}}, Nothing}, Vector{Float64}}, Matrix{Float64}}}, NeuralPDE.PINNRepresentation, Bool, Vector{Int64}, Int64, NeuralPDE.Phi{Optimisers.Restructure{Flux.Chain{Tuple{Flux.Dense{typeof(σ), Matrix{Float64}, Vector{Float64}}, Flux.Dense{typeof(σ), Matrix{Float64}, Vector{Float64}}, Flux.Dense{typeof(identity), Matrix{Float64}, Vector{Float64}}}}, NamedTuple{(:layers,), Tuple{Tuple{NamedTuple{(:weight, :bias, :σ), Tuple{Int64, Int64, Tuple{}}}, NamedTuple{(:weight, :bias, :σ), Tuple{Int64, Int64, Tuple{}}}, NamedTuple{(:weight, :bias, :σ), Tuple{Int64, Int64, Tuple{}}}}}}}, Nothing}, typeof(additional_loss), Bool, Vector{Float64}}, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, typeof(SciMLBase.DEFAULT_OBSERVED_NO_TIME), Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing}, Optimization.ReInitCache{Vector{Float64}, SciMLBase.NullParameters}}}, Optimization.var"#231#240"{Optimization.var"#227#236"{OptimizationFunction{true, Optimization.AutoZygote, NeuralPDE.var"#full_loss_function#308"{NeuralPDE.var"#null_nonadaptive_loss#115", Vector{NeuralPDE.var"#74#75"{NeuralPDE.var"#209#210"{RuntimeGeneratedFunctions.RuntimeGeneratedFunction{(:cord, Symbol("##θ#293"), :phi, :derivative, :integral, :u, :p), NeuralPDE.var"#_RGF_ModTag", NeuralPDE.var"#_RGF_ModTag", (0x03d798f9, 0x34fa0c63, 0xe7b60143, 0x8b2612da, 0x6b36183f), Expr}, NeuralPDE.var"#12#13", NeuralPDE.var"#269#276"{NeuralPDE.var"#269#270#277"{typeof(NeuralPDE.numeric_derivative)}, Dict{Symbol, Int64}, Dict{Symbol, Int64}, GridTraining{Float64}}, typeof(NeuralPDE.numeric_derivative), NeuralPDE.Phi{Optimisers.Restructure{Flux.Chain{Tuple{Flux.Dense{typeof(σ), Matrix{Float64}, Vector{Float64}}, Flux.Dense{typeof(σ), Matrix{Float64}, Vector{Float64}}, Flux.Dense{typeof(identity), Matrix{Float64}, Vector{Float64}}}}, NamedTuple{(:layers,), Tuple{Tuple{NamedTuple{(:weight, :bias, :σ), Tuple{Int64, Int64, Tuple{}}}, NamedTuple{(:weight, :bias, :σ), Tuple{Int64, Int64, Tuple{}}}, NamedTuple{(:weight, :bias, :σ), Tuple{Int64, Int64, Tuple{}}}}}}}, Nothing}, Vector{Float64}}, Matrix{Float64}}}, Vector{NeuralPDE.var"#74#75"{NeuralPDE.var"#209#210"{RuntimeGeneratedFunctions.RuntimeGeneratedFunction{(:cord, Symbol("##θ#293"), :phi, :derivative, :integral, :u, :p), NeuralPDE.var"#_RGF_ModTag", NeuralPDE.var"#_RGF_ModTag", (0x64128365, 0xe99c13c8, 0x3553718d, 0xc14f6af0, 0x258a1b45), Expr}, NeuralPDE.var"#12#13", NeuralPDE.var"#269#276"{NeuralPDE.var"#269#270#277"{typeof(NeuralPDE.numeric_derivative)}, Dict{Symbol, Int64}, Dict{Symbol, Int64}, GridTraining{Float64}}, typeof(NeuralPDE.numeric_derivative), NeuralPDE.Phi{Optimisers.Restructure{Flux.Chain{Tuple{Flux.Dense{typeof(σ), Matrix{Float64}, Vector{Float64}}, Flux.Dense{typeof(σ), Matrix{Float64}, Vector{Float64}}, Flux.Dense{typeof(identity), Matrix{Float64}, Vector{Float64}}}}, NamedTuple{(:layers,), Tuple{Tuple{NamedTuple{(:weight, :bias, :σ), Tuple{Int64, Int64, Tuple{}}}, NamedTuple{(:weight, :bias, :σ), Tuple{Int64, Int64, Tuple{}}}, NamedTuple{(:weight, :bias, :σ), Tuple{Int64, Int64, Tuple{}}}}}}}, Nothing}, Vector{Float64}}, Matrix{Float64}}}, NeuralPDE.PINNRepresentation, Bool, Vector{Int64}, Int64, NeuralPDE.Phi{Optimisers.Restructure{Flux.Chain{Tuple{Flux.Dense{typeof(σ), Matrix{Float64}, Vector{Float64}}, Flux.Dense{typeof(σ), Matrix{Float64}, Vector{Float64}}, Flux.Dense{typeof(identity), Matrix{Float64}, Vector{Float64}}}}, NamedTuple{(:layers,), Tuple{Tuple{NamedTuple{(:weight, :bias, :σ), Tuple{Int64, Int64, Tuple{}}}, NamedTuple{(:weight, :bias, :σ), Tuple{Int64, Int64, Tuple{}}}, NamedTuple{(:weight, :bias, :σ), Tuple{Int64, Int64, Tuple{}}}}}}}, Nothing}, typeof(additional_loss), Bool, Vector{Float64}}, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, typeof(SciMLBase.DEFAULT_OBSERVED_NO_TIME), Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing}, Optimization.ReInitCache{Vector{Float64}, SciMLBase.NullParameters}}}, Optimization.var"#235#244", Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, typeof(SciMLBase.DEFAULT_OBSERVED_NO_TIME), Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing}, Optimization.ReInitCache{Vector{Float64}, SciMLBase.NullParameters}, Nothing, Nothing, Nothing, Nothing, Nothing, BFGS{LineSearches.InitialStatic{Float64}, LineSearches.HagerZhang{Float64, Base.RefValue{Bool}}, Nothing, Nothing, Flat}, Base.Iterators.Cycle{Tuple{Optimization.NullData}}, Bool, OptimizationOptimJL.var"#6#8"}, OptimizationOptimJL.var"#9#15"{OptimizationOptimJL.OptimJLOptimizationCache{OptimizationFunction{false, Optimization.AutoZygote, OptimizationFunction{true, Optimization.AutoZygote, NeuralPDE.var"#full_loss_function#308"{NeuralPDE.var"#null_nonadaptive_loss#115", Vector{NeuralPDE.var"#74#75"{NeuralPDE.var"#209#210"{RuntimeGeneratedFunctions.RuntimeGeneratedFunction{(:cord, Symbol("##θ#293"), :phi, :derivative, :integral, :u, :p), NeuralPDE.var"#_RGF_ModTag", NeuralPDE.var"#_RGF_ModTag", (0x03d798f9, 0x34fa0c63, 0xe7b60143, 0x8b2612da, 0x6b36183f), Expr}, NeuralPDE.var"#12#13", NeuralPDE.var"#269#276"{NeuralPDE.var"#269#270#277"{typeof(NeuralPDE.numeric_derivative)}, Dict{Symbol, Int64}, Dict{Symbol, Int64}, GridTraining{Float64}}, typeof(NeuralPDE.numeric_derivative), NeuralPDE.Phi{Optimisers.Restructure{Flux.Chain{Tuple{Flux.Dense{typeof(σ), Matrix{Float64}, Vector{Float64}}, Flux.Dense{typeof(σ), Matrix{Float64}, Vector{Float64}}, Flux.Dense{typeof(identity), Matrix{Float64}, Vector{Float64}}}}, NamedTuple{(:layers,), Tuple{Tuple{NamedTuple{(:weight, :bias, :σ), Tuple{Int64, Int64, Tuple{}}}, NamedTuple{(:weight, :bias, :σ), Tuple{Int64, Int64, Tuple{}}}, NamedTuple{(:weight, :bias, :σ), Tuple{Int64, Int64, Tuple{}}}}}}}, Nothing}, Vector{Float64}}, Matrix{Float64}}}, Vector{NeuralPDE.var"#74#75"{NeuralPDE.var"#209#210"{RuntimeGeneratedFunctions.RuntimeGeneratedFunction{(:cord, Symbol("##θ#293"), :phi, :derivative, :integral, :u, :p), NeuralPDE.var"#_RGF_ModTag", NeuralPDE.var"#_RGF_ModTag", (0x64128365, 0xe99c13c8, 0x3553718d, 0xc14f6af0, 0x258a1b45), Expr}, NeuralPDE.var"#12#13", NeuralPDE.var"#269#276"{NeuralPDE.var"#269#270#277"{typeof(NeuralPDE.numeric_derivative)}, Dict{Symbol, Int64}, Dict{Symbol, Int64}, GridTraining{Float64}}, typeof(NeuralPDE.numeric_derivative), NeuralPDE.Phi{Optimisers.Restructure{Flux.Chain{Tuple{Flux.Dense{typeof(σ), Matrix{Float64}, Vector{Float64}}, Flux.Dense{typeof(σ), Matrix{Float64}, Vector{Float64}}, Flux.Dense{typeof(identity), Matrix{Float64}, Vector{Float64}}}}, NamedTuple{(:layers,), Tuple{Tuple{NamedTuple{(:weight, :bias, :σ), Tuple{Int64, Int64, Tuple{}}}, NamedTuple{(:weight, :bias, :σ), Tuple{Int64, Int64, Tuple{}}}, NamedTuple{(:weight, :bias, :σ), Tuple{Int64, Int64, Tuple{}}}}}}}, Nothing}, Vector{Float64}}, Matrix{Float64}}}, NeuralPDE.PINNRepresentation, Bool, Vector{Int64}, Int64, NeuralPDE.Phi{Optimisers.Restructure{Flux.Chain{Tuple{Flux.Dense{typeof(σ), Matrix{Float64}, Vector{Float64}}, Flux.Dense{typeof(σ), Matrix{Float64}, Vector{Float64}}, Flux.Dense{typeof(identity), Matrix{Float64}, Vector{Float64}}}}, NamedTuple{(:layers,), Tuple{Tuple{NamedTuple{(:weight, :bias, :σ), Tuple{Int64, Int64, Tuple{}}}, NamedTuple{(:weight, :bias, :σ), Tuple{Int64, Int64, Tuple{}}}, NamedTuple{(:weight, :bias, :σ), Tuple{Int64, Int64, Tuple{}}}}}}}, Nothing}, typeof(additional_loss), Bool, Vector{Float64}}, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, typeof(SciMLBase.DEFAULT_OBSERVED_NO_TIME), Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing}, Optimization.var"#228#237"{Optimization.var"#227#236"{OptimizationFunction{true, Optimization.AutoZygote, NeuralPDE.var"#full_loss_function#308"{NeuralPDE.var"#null_nonadaptive_loss#115", Vector{NeuralPDE.var"#74#75"{NeuralPDE.var"#209#210"{RuntimeGeneratedFunctions.RuntimeGeneratedFunction{(:cord, Symbol("##θ#293"), :phi, :derivative, :integral, :u, :p), NeuralPDE.var"#_RGF_ModTag", NeuralPDE.var"#_RGF_ModTag", (0x03d798f9, 0x34fa0c63, 0xe7b60143, 0x8b2612da, 0x6b36183f), Expr}, NeuralPDE.var"#12#13", NeuralPDE.var"#269#276"{NeuralPDE.var"#269#270#277"{typeof(NeuralPDE.numeric_derivative)}, Dict{Symbol, Int64}, Dict{Symbol, Int64}, GridTraining{Float64}}, typeof(NeuralPDE.numeric_derivative), NeuralPDE.Phi{Optimisers.Restructure{Flux.Chain{Tuple{Flux.Dense{typeof(σ), Matrix{Float64}, Vector{Float64}}, Flux.Dense{typeof(σ), Matrix{Float64}, Vector{Float64}}, Flux.Dense{typeof(identity), Matrix{Float64}, Vector{Float64}}}}, NamedTuple{(:layers,), Tuple{Tuple{NamedTuple{(:weight, :bias, :σ), Tuple{Int64, Int64, Tuple{}}}, NamedTuple{(:weight, :bias, :σ), Tuple{Int64, Int64, Tuple{}}}, NamedTuple{(:weight, :bias, :σ), Tuple{Int64, Int64, Tuple{}}}}}}}, Nothing}, Vector{Float64}}, Matrix{Float64}}}, Vector{NeuralPDE.var"#74#75"{NeuralPDE.var"#209#210"{RuntimeGeneratedFunctions.RuntimeGeneratedFunction{(:cord, Symbol("##θ#293"), :phi, :derivative, :integral, :u, :p), NeuralPDE.var"#_RGF_ModTag", NeuralPDE.var"#_RGF_ModTag", (0x64128365, 0xe99c13c8, 0x3553718d, 0xc14f6af0, 0x258a1b45), Expr}, NeuralPDE.var"#12#13", NeuralPDE.var"#269#276"{NeuralPDE.var"#269#270#277"{typeof(NeuralPDE.numeric_derivative)}, Dict{Symbol, Int64}, Dict{Symbol, Int64}, GridTraining{Float64}}, typeof(NeuralPDE.numeric_derivative), NeuralPDE.Phi{Optimisers.Restructure{Flux.Chain{Tuple{Flux.Dense{typeof(σ), Matrix{Float64}, Vector{Float64}}, Flux.Dense{typeof(σ), Matrix{Float64}, Vector{Float64}}, Flux.Dense{typeof(identity), Matrix{Float64}, Vector{Float64}}}}, NamedTuple{(:layers,), Tuple{Tuple{NamedTuple{(:weight, :bias, :σ), Tuple{Int64, Int64, Tuple{}}}, NamedTuple{(:weight, :bias, :σ), Tuple{Int64, Int64, Tuple{}}}, NamedTuple{(:weight, :bias, :σ), Tuple{Int64, Int64, Tuple{}}}}}}}, Nothing}, Vector{Float64}}, Matrix{Float64}}}, NeuralPDE.PINNRepresentation, Bool, Vector{Int64}, Int64, NeuralPDE.Phi{Optimisers.Restructure{Flux.Chain{Tuple{Flux.Dense{typeof(σ), Matrix{Float64}, Vector{Float64}}, Flux.Dense{typeof(σ), Matrix{Float64}, Vector{Float64}}, Flux.Dense{typeof(identity), Matrix{Float64}, Vector{Float64}}}}, NamedTuple{(:layers,), Tuple{Tuple{NamedTuple{(:weight, :bias, :σ), Tuple{Int64, Int64, Tuple{}}}, NamedTuple{(:weight, :bias, :σ), Tuple{Int64, Int64, Tuple{}}}, NamedTuple{(:weight, :bias, :σ), Tuple{Int64, Int64, Tuple{}}}}}}}, Nothing}, typeof(additional_loss), Bool, Vector{Float64}}, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, typeof(SciMLBase.DEFAULT_OBSERVED_NO_TIME), Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing}, Optimization.ReInitCache{Vector{Float64}, SciMLBase.NullParameters}}}, Optimization.var"#231#240"{Optimization.var"#227#236"{OptimizationFunction{true, Optimization.AutoZygote, NeuralPDE.var"#full_loss_function#308"{NeuralPDE.var"#null_nonadaptive_loss#115", Vector{NeuralPDE.var"#74#75"{NeuralPDE.var"#209#210"{RuntimeGeneratedFunctions.RuntimeGeneratedFunction{(:cord, Symbol("##θ#293"), :phi, :derivative, :integral, :u, :p), NeuralPDE.var"#_RGF_ModTag", NeuralPDE.var"#_RGF_ModTag", (0x03d798f9, 0x34fa0c63, 0xe7b60143, 0x8b2612da, 0x6b36183f), Expr}, NeuralPDE.var"#12#13", NeuralPDE.var"#269#276"{NeuralPDE.var"#269#270#277"{typeof(NeuralPDE.numeric_derivative)}, Dict{Symbol, Int64}, Dict{Symbol, Int64}, GridTraining{Float64}}, typeof(NeuralPDE.numeric_derivative), NeuralPDE.Phi{Optimisers.Restructure{Flux.Chain{Tuple{Flux.Dense{typeof(σ), Matrix{Float64}, Vector{Float64}}, Flux.Dense{typeof(σ), Matrix{Float64}, Vector{Float64}}, Flux.Dense{typeof(identity), Matrix{Float64}, Vector{Float64}}}}, NamedTuple{(:layers,), Tuple{Tuple{NamedTuple{(:weight, :bias, :σ), Tuple{Int64, Int64, Tuple{}}}, NamedTuple{(:weight, :bias, :σ), Tuple{Int64, Int64, Tuple{}}}, NamedTuple{(:weight, :bias, :σ), Tuple{Int64, Int64, Tuple{}}}}}}}, Nothing}, Vector{Float64}}, Matrix{Float64}}}, Vector{NeuralPDE.var"#74#75"{NeuralPDE.var"#209#210"{RuntimeGeneratedFunctions.RuntimeGeneratedFunction{(:cord, Symbol("##θ#293"), :phi, :derivative, :integral, :u, :p), NeuralPDE.var"#_RGF_ModTag", NeuralPDE.var"#_RGF_ModTag", (0x64128365, 0xe99c13c8, 0x3553718d, 0xc14f6af0, 0x258a1b45), Expr}, NeuralPDE.var"#12#13", NeuralPDE.var"#269#276"{NeuralPDE.var"#269#270#277"{typeof(NeuralPDE.numeric_derivative)}, Dict{Symbol, Int64}, Dict{Symbol, Int64}, GridTraining{Float64}}, typeof(NeuralPDE.numeric_derivative), NeuralPDE.Phi{Optimisers.Restructure{Flux.Chain{Tuple{Flux.Dense{typeof(σ), Matrix{Float64}, Vector{Float64}}, Flux.Dense{typeof(σ), Matrix{Float64}, Vector{Float64}}, Flux.Dense{typeof(identity), Matrix{Float64}, Vector{Float64}}}}, NamedTuple{(:layers,), Tuple{Tuple{NamedTuple{(:weight, :bias, :σ), Tuple{Int64, Int64, Tuple{}}}, NamedTuple{(:weight, :bias, :σ), Tuple{Int64, Int64, Tuple{}}}, NamedTuple{(:weight, :bias, :σ), Tuple{Int64, Int64, Tuple{}}}}}}}, Nothing}, Vector{Float64}}, Matrix{Float64}}}, NeuralPDE.PINNRepresentation, Bool, Vector{Int64}, Int64, NeuralPDE.Phi{Optimisers.Restructure{Flux.Chain{Tuple{Flux.Dense{typeof(σ), Matrix{Float64}, Vector{Float64}}, Flux.Dense{typeof(σ), Matrix{Float64}, Vector{Float64}}, Flux.Dense{typeof(identity), Matrix{Float64}, Vector{Float64}}}}, NamedTuple{(:layers,), Tuple{Tuple{NamedTuple{(:weight, :bias, :σ), Tuple{Int64, Int64, Tuple{}}}, NamedTuple{(:weight, :bias, :σ), Tuple{Int64, Int64, Tuple{}}}, NamedTuple{(:weight, :bias, :σ), Tuple{Int64, Int64, Tuple{}}}}}}}, Nothing}, typeof(additional_loss), Bool, Vector{Float64}}, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, typeof(SciMLBase.DEFAULT_OBSERVED_NO_TIME), Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing}, Optimization.ReInitCache{Vector{Float64}, SciMLBase.NullParameters}}}, Optimization.var"#235#244", Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, typeof(SciMLBase.DEFAULT_OBSERVED_NO_TIME), Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing}, Optimization.ReInitCache{Vector{Float64}, SciMLBase.NullParameters}, Nothing, Nothing, Nothing, Nothing, Nothing, BFGS{LineSearches.InitialStatic{Float64}, LineSearches.HagerZhang{Float64, Base.RefValue{Bool}}, Nothing, Nothing, Flat}, Base.Iterators.Cycle{Tuple{Optimization.NullData}}, Bool, OptimizationOptimJL.var"#6#8"}}})(G::Vector{Float64}, θ::Vector{Float64})
    @ OptimizationOptimJL C:\Users\loren\.julia\packages\OptimizationOptimJL\uRfW9\src\OptimizationOptimJL.jl:200
 [61] value_gradient!!(obj::TwiceDifferentiable{Float64, Vector{Float64}, Matrix{Float64}, Vector{Float64}}, x::Vector{Float64})
    @ NLSolversBase C:\Users\loren\.julia\packages\NLSolversBase\kavn7\src\interface.jl:82
 [62] initial_state(method::BFGS{LineSearches.InitialStatic{Float64}, LineSearches.HagerZhang{Float64, Base.RefValue{Bool}}, Nothing, Nothing, Flat}, options::Optim.Options{Float64, OptimizationOptimJL.var"#_cb#14"{OptimizationOptimJL.OptimJLOptimizationCache{OptimizationFunction{false, Optimization.AutoZygote, OptimizationFunction{true, Optimization.AutoZygote, NeuralPDE.var"#full_loss_function#308"{NeuralPDE.var"#null_nonadaptive_loss#115", Vector{NeuralPDE.var"#74#75"{NeuralPDE.var"#209#210"{RuntimeGeneratedFunctions.RuntimeGeneratedFunction{(:cord, Symbol("##θ#293"), :phi, :derivative, :integral, :u, :p), NeuralPDE.var"#_RGF_ModTag", NeuralPDE.var"#_RGF_ModTag", (0x03d798f9, 0x34fa0c63, 0xe7b60143, 0x8b2612da, 0x6b36183f), Expr}, NeuralPDE.var"#12#13", NeuralPDE.var"#269#276"{NeuralPDE.var"#269#270#277"{typeof(NeuralPDE.numeric_derivative)}, Dict{Symbol, Int64}, Dict{Symbol, Int64}, GridTraining{Float64}}, typeof(NeuralPDE.numeric_derivative), NeuralPDE.Phi{Optimisers.Restructure{Flux.Chain{Tuple{Flux.Dense{typeof(σ), Matrix{Float64}, Vector{Float64}}, Flux.Dense{typeof(σ), Matrix{Float64}, Vector{Float64}}, Flux.Dense{typeof(identity), Matrix{Float64}, Vector{Float64}}}}, NamedTuple{(:layers,), Tuple{Tuple{NamedTuple{(:weight, :bias, :σ), Tuple{Int64, Int64, Tuple{}}}, NamedTuple{(:weight, :bias, :σ), Tuple{Int64, Int64, Tuple{}}}, NamedTuple{(:weight, :bias, :σ), Tuple{Int64, Int64, Tuple{}}}}}}}, Nothing}, Vector{Float64}}, Matrix{Float64}}}, Vector{NeuralPDE.var"#74#75"{NeuralPDE.var"#209#210"{RuntimeGeneratedFunctions.RuntimeGeneratedFunction{(:cord, Symbol("##θ#293"), :phi, :derivative, :integral, :u, :p), NeuralPDE.var"#_RGF_ModTag", NeuralPDE.var"#_RGF_ModTag", (0x64128365, 0xe99c13c8, 0x3553718d, 0xc14f6af0, 0x258a1b45), Expr}, NeuralPDE.var"#12#13", NeuralPDE.var"#269#276"{NeuralPDE.var"#269#270#277"{typeof(NeuralPDE.numeric_derivative)}, Dict{Symbol, Int64}, Dict{Symbol, Int64}, GridTraining{Float64}}, typeof(NeuralPDE.numeric_derivative), NeuralPDE.Phi{Optimisers.Restructure{Flux.Chain{Tuple{Flux.Dense{typeof(σ), Matrix{Float64}, Vector{Float64}}, Flux.Dense{typeof(σ), Matrix{Float64}, Vector{Float64}}, Flux.Dense{typeof(identity), Matrix{Float64}, Vector{Float64}}}}, NamedTuple{(:layers,), Tuple{Tuple{NamedTuple{(:weight, :bias, :σ), Tuple{Int64, Int64, Tuple{}}}, NamedTuple{(:weight, :bias, :σ), Tuple{Int64, Int64, Tuple{}}}, NamedTuple{(:weight, :bias, :σ), Tuple{Int64, Int64, Tuple{}}}}}}}, Nothing}, Vector{Float64}}, Matrix{Float64}}}, NeuralPDE.PINNRepresentation, Bool, Vector{Int64}, Int64, NeuralPDE.Phi{Optimisers.Restructure{Flux.Chain{Tuple{Flux.Dense{typeof(σ), Matrix{Float64}, Vector{Float64}}, Flux.Dense{typeof(σ), Matrix{Float64}, Vector{Float64}}, Flux.Dense{typeof(identity), Matrix{Float64}, Vector{Float64}}}}, NamedTuple{(:layers,), Tuple{Tuple{NamedTuple{(:weight, :bias, :σ), Tuple{Int64, Int64, Tuple{}}}, NamedTuple{(:weight, :bias, :σ), Tuple{Int64, Int64, Tuple{}}}, NamedTuple{(:weight, :bias, :σ), Tuple{Int64, Int64, Tuple{}}}}}}}, Nothing}, typeof(additional_loss), Bool, Vector{Float64}}, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, typeof(SciMLBase.DEFAULT_OBSERVED_NO_TIME), Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing}, Optimization.var"#228#237"{Optimization.var"#227#236"{OptimizationFunction{true, Optimization.AutoZygote, NeuralPDE.var"#full_loss_function#308"{NeuralPDE.var"#null_nonadaptive_loss#115", Vector{NeuralPDE.var"#74#75"{NeuralPDE.var"#209#210"{RuntimeGeneratedFunctions.RuntimeGeneratedFunction{(:cord, Symbol("##θ#293"), :phi, :derivative, :integral, :u, :p), NeuralPDE.var"#_RGF_ModTag", NeuralPDE.var"#_RGF_ModTag", (0x03d798f9, 0x34fa0c63, 0xe7b60143, 0x8b2612da, 0x6b36183f), Expr}, NeuralPDE.var"#12#13", NeuralPDE.var"#269#276"{NeuralPDE.var"#269#270#277"{typeof(NeuralPDE.numeric_derivative)}, Dict{Symbol, Int64}, Dict{Symbol, Int64}, GridTraining{Float64}}, typeof(NeuralPDE.numeric_derivative), NeuralPDE.Phi{Optimisers.Restructure{Flux.Chain{Tuple{Flux.Dense{typeof(σ), Matrix{Float64}, Vector{Float64}}, Flux.Dense{typeof(σ), Matrix{Float64}, Vector{Float64}}, Flux.Dense{typeof(identity), Matrix{Float64}, Vector{Float64}}}}, NamedTuple{(:layers,), Tuple{Tuple{NamedTuple{(:weight, :bias, :σ), Tuple{Int64, Int64, Tuple{}}}, NamedTuple{(:weight, :bias, :σ), Tuple{Int64, Int64, Tuple{}}}, NamedTuple{(:weight, :bias, :σ), Tuple{Int64, Int64, Tuple{}}}}}}}, Nothing}, Vector{Float64}}, Matrix{Float64}}}, Vector{NeuralPDE.var"#74#75"{NeuralPDE.var"#209#210"{RuntimeGeneratedFunctions.RuntimeGeneratedFunction{(:cord, Symbol("##θ#293"), :phi, :derivative, :integral, :u, :p), NeuralPDE.var"#_RGF_ModTag", NeuralPDE.var"#_RGF_ModTag", (0x64128365, 0xe99c13c8, 0x3553718d, 0xc14f6af0, 0x258a1b45), Expr}, NeuralPDE.var"#12#13", NeuralPDE.var"#269#276"{NeuralPDE.var"#269#270#277"{typeof(NeuralPDE.numeric_derivative)}, Dict{Symbol, Int64}, Dict{Symbol, Int64}, GridTraining{Float64}}, typeof(NeuralPDE.numeric_derivative), NeuralPDE.Phi{Optimisers.Restructure{Flux.Chain{Tuple{Flux.Dense{typeof(σ), Matrix{Float64}, Vector{Float64}}, Flux.Dense{typeof(σ), Matrix{Float64}, Vector{Float64}}, Flux.Dense{typeof(identity), Matrix{Float64}, Vector{Float64}}}}, NamedTuple{(:layers,), Tuple{Tuple{NamedTuple{(:weight, :bias, :σ), Tuple{Int64, Int64, Tuple{}}}, NamedTuple{(:weight, :bias, :σ), Tuple{Int64, Int64, Tuple{}}}, NamedTuple{(:weight, :bias, :σ), Tuple{Int64, Int64, Tuple{}}}}}}}, Nothing}, Vector{Float64}}, Matrix{Float64}}}, NeuralPDE.PINNRepresentation, Bool, Vector{Int64}, Int64, NeuralPDE.Phi{Optimisers.Restructure{Flux.Chain{Tuple{Flux.Dense{typeof(σ), Matrix{Float64}, Vector{Float64}}, Flux.Dense{typeof(σ), Matrix{Float64}, Vector{Float64}}, Flux.Dense{typeof(identity), Matrix{Float64}, Vector{Float64}}}}, NamedTuple{(:layers,), Tuple{Tuple{NamedTuple{(:weight, :bias, :σ), Tuple{Int64, Int64, Tuple{}}}, NamedTuple{(:weight, :bias, :σ), Tuple{Int64, Int64, Tuple{}}}, NamedTuple{(:weight, :bias, :σ), Tuple{Int64, Int64, Tuple{}}}}}}}, Nothing}, typeof(additional_loss), Bool, Vector{Float64}}, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, typeof(SciMLBase.DEFAULT_OBSERVED_NO_TIME), Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing}, Optimization.ReInitCache{Vector{Float64}, SciMLBase.NullParameters}}}, Optimization.var"#231#240"{Optimization.var"#227#236"{OptimizationFunction{true, Optimization.AutoZygote, NeuralPDE.var"#full_loss_function#308"{NeuralPDE.var"#null_nonadaptive_loss#115", Vector{NeuralPDE.var"#74#75"{NeuralPDE.var"#209#210"{RuntimeGeneratedFunctions.RuntimeGeneratedFunction{(:cord, Symbol("##θ#293"), :phi, :derivative, :integral, :u, :p), NeuralPDE.var"#_RGF_ModTag", NeuralPDE.var"#_RGF_ModTag", (0x03d798f9, 0x34fa0c63, 0xe7b60143, 0x8b2612da, 0x6b36183f), Expr}, NeuralPDE.var"#12#13", NeuralPDE.var"#269#276"{NeuralPDE.var"#269#270#277"{typeof(NeuralPDE.numeric_derivative)}, Dict{Symbol, Int64}, Dict{Symbol, Int64}, GridTraining{Float64}}, typeof(NeuralPDE.numeric_derivative), NeuralPDE.Phi{Optimisers.Restructure{Flux.Chain{Tuple{Flux.Dense{typeof(σ), Matrix{Float64}, Vector{Float64}}, Flux.Dense{typeof(σ), Matrix{Float64}, Vector{Float64}}, Flux.Dense{typeof(identity), Matrix{Float64}, Vector{Float64}}}}, NamedTuple{(:layers,), Tuple{Tuple{NamedTuple{(:weight, :bias, :σ), Tuple{Int64, Int64, Tuple{}}}, NamedTuple{(:weight, :bias, :σ), Tuple{Int64, Int64, Tuple{}}}, NamedTuple{(:weight, :bias, :σ), Tuple{Int64, Int64, Tuple{}}}}}}}, Nothing}, Vector{Float64}}, Matrix{Float64}}}, Vector{NeuralPDE.var"#74#75"{NeuralPDE.var"#209#210"{RuntimeGeneratedFunctions.RuntimeGeneratedFunction{(:cord, Symbol("##θ#293"), :phi, :derivative, :integral, :u, :p), NeuralPDE.var"#_RGF_ModTag", NeuralPDE.var"#_RGF_ModTag", (0x64128365, 0xe99c13c8, 0x3553718d, 0xc14f6af0, 0x258a1b45), Expr}, NeuralPDE.var"#12#13", NeuralPDE.var"#269#276"{NeuralPDE.var"#269#270#277"{typeof(NeuralPDE.numeric_derivative)}, Dict{Symbol, Int64}, Dict{Symbol, Int64}, GridTraining{Float64}}, typeof(NeuralPDE.numeric_derivative), NeuralPDE.Phi{Optimisers.Restructure{Flux.Chain{Tuple{Flux.Dense{typeof(σ), Matrix{Float64}, Vector{Float64}}, Flux.Dense{typeof(σ), Matrix{Float64}, Vector{Float64}}, Flux.Dense{typeof(identity), Matrix{Float64}, Vector{Float64}}}}, NamedTuple{(:layers,), Tuple{Tuple{NamedTuple{(:weight, :bias, :σ), Tuple{Int64, Int64, Tuple{}}}, NamedTuple{(:weight, :bias, :σ), Tuple{Int64, Int64, Tuple{}}}, NamedTuple{(:weight, :bias, :σ), Tuple{Int64, Int64, Tuple{}}}}}}}, Nothing}, Vector{Float64}}, Matrix{Float64}}}, NeuralPDE.PINNRepresentation, Bool, Vector{Int64}, Int64, NeuralPDE.Phi{Optimisers.Restructure{Flux.Chain{Tuple{Flux.Dense{typeof(σ), Matrix{Float64}, Vector{Float64}}, Flux.Dense{typeof(σ), Matrix{Float64}, Vector{Float64}}, Flux.Dense{typeof(identity), Matrix{Float64}, Vector{Float64}}}}, NamedTuple{(:layers,), Tuple{Tuple{NamedTuple{(:weight, :bias, :σ), Tuple{Int64, Int64, Tuple{}}}, NamedTuple{(:weight, :bias, :σ), Tuple{Int64, Int64, Tuple{}}}, NamedTuple{(:weight, :bias, :σ), Tuple{Int64, Int64, Tuple{}}}}}}}, Nothing}, typeof(additional_loss), Bool, Vector{Float64}}, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, typeof(SciMLBase.DEFAULT_OBSERVED_NO_TIME), Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing}, Optimization.ReInitCache{Vector{Float64}, SciMLBase.NullParameters}}}, Optimization.var"#235#244", Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, typeof(SciMLBase.DEFAULT_OBSERVED_NO_TIME), Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing}, Optimization.ReInitCache{Vector{Float64}, SciMLBase.NullParameters}, Nothing, Nothing, Nothing, Nothing, Nothing, BFGS{LineSearches.InitialStatic{Float64}, LineSearches.HagerZhang{Float64, Base.RefValue{Bool}}, Nothing, Nothing, Flat}, Base.Iterators.Cycle{Tuple{Optimization.NullData}}, Bool, OptimizationOptimJL.var"#6#8"}}}, d::TwiceDifferentiable{Float64, Vector{Float64}, Matrix{Float64}, Vector{Float64}}, initial_x::Vector{Float64})
    @ Optim C:\Users\loren\.julia\packages\Optim\V8ZEC\src\multivariate\solvers\first_order\bfgs.jl:94
 [63] optimize(d::TwiceDifferentiable{Float64, Vector{Float64}, Matrix{Float64}, Vector{Float64}}, initial_x::Vector{Float64}, method::BFGS{LineSearches.InitialStatic{Float64}, LineSearches.HagerZhang{Float64, Base.RefValue{Bool}}, Nothing, Nothing, Flat}, options::Optim.Options{Float64, OptimizationOptimJL.var"#_cb#14"{OptimizationOptimJL.OptimJLOptimizationCache{OptimizationFunction{false, Optimization.AutoZygote, OptimizationFunction{true, Optimization.AutoZygote, NeuralPDE.var"#full_loss_function#308"{NeuralPDE.var"#null_nonadaptive_loss#115", Vector{NeuralPDE.var"#74#75"{NeuralPDE.var"#209#210"{RuntimeGeneratedFunctions.RuntimeGeneratedFunction{(:cord, Symbol("##θ#293"), :phi, :derivative, :integral, :u, :p), NeuralPDE.var"#_RGF_ModTag", NeuralPDE.var"#_RGF_ModTag", (0x03d798f9, 0x34fa0c63, 0xe7b60143, 0x8b2612da, 0x6b36183f), Expr}, NeuralPDE.var"#12#13", NeuralPDE.var"#269#276"{NeuralPDE.var"#269#270#277"{typeof(NeuralPDE.numeric_derivative)}, Dict{Symbol, Int64}, Dict{Symbol, Int64}, GridTraining{Float64}}, typeof(NeuralPDE.numeric_derivative), NeuralPDE.Phi{Optimisers.Restructure{Flux.Chain{Tuple{Flux.Dense{typeof(σ), Matrix{Float64}, Vector{Float64}}, Flux.Dense{typeof(σ), Matrix{Float64}, Vector{Float64}}, Flux.Dense{typeof(identity), Matrix{Float64}, Vector{Float64}}}}, NamedTuple{(:layers,), Tuple{Tuple{NamedTuple{(:weight, :bias, :σ), Tuple{Int64, Int64, Tuple{}}}, NamedTuple{(:weight, :bias, :σ), Tuple{Int64, Int64, Tuple{}}}, NamedTuple{(:weight, :bias, :σ), Tuple{Int64, Int64, Tuple{}}}}}}}, Nothing}, Vector{Float64}}, Matrix{Float64}}}, Vector{NeuralPDE.var"#74#75"{NeuralPDE.var"#209#210"{RuntimeGeneratedFunctions.RuntimeGeneratedFunction{(:cord, Symbol("##θ#293"), :phi, :derivative, :integral, :u, :p), NeuralPDE.var"#_RGF_ModTag", NeuralPDE.var"#_RGF_ModTag", (0x64128365, 0xe99c13c8, 0x3553718d, 0xc14f6af0, 0x258a1b45), Expr}, NeuralPDE.var"#12#13", NeuralPDE.var"#269#276"{NeuralPDE.var"#269#270#277"{typeof(NeuralPDE.numeric_derivative)}, Dict{Symbol, Int64}, Dict{Symbol, Int64}, GridTraining{Float64}}, typeof(NeuralPDE.numeric_derivative), NeuralPDE.Phi{Optimisers.Restructure{Flux.Chain{Tuple{Flux.Dense{typeof(σ), Matrix{Float64}, Vector{Float64}}, Flux.Dense{typeof(σ), Matrix{Float64}, Vector{Float64}}, Flux.Dense{typeof(identity), Matrix{Float64}, Vector{Float64}}}}, NamedTuple{(:layers,), Tuple{Tuple{NamedTuple{(:weight, :bias, :σ), Tuple{Int64, Int64, Tuple{}}}, NamedTuple{(:weight, :bias, :σ), Tuple{Int64, Int64, Tuple{}}}, NamedTuple{(:weight, :bias, :σ), Tuple{Int64, Int64, Tuple{}}}}}}}, Nothing}, Vector{Float64}}, Matrix{Float64}}}, NeuralPDE.PINNRepresentation, Bool, Vector{Int64}, Int64, NeuralPDE.Phi{Optimisers.Restructure{Flux.Chain{Tuple{Flux.Dense{typeof(σ), Matrix{Float64}, Vector{Float64}}, Flux.Dense{typeof(σ), Matrix{Float64}, Vector{Float64}}, Flux.Dense{typeof(identity), Matrix{Float64}, Vector{Float64}}}}, NamedTuple{(:layers,), Tuple{Tuple{NamedTuple{(:weight, :bias, :σ), Tuple{Int64, Int64, Tuple{}}}, NamedTuple{(:weight, :bias, :σ), Tuple{Int64, Int64, Tuple{}}}, NamedTuple{(:weight, :bias, :σ), Tuple{Int64, Int64, Tuple{}}}}}}}, Nothing}, typeof(additional_loss), Bool, Vector{Float64}}, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, typeof(SciMLBase.DEFAULT_OBSERVED_NO_TIME), Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing}, Optimization.var"#228#237"{Optimization.var"#227#236"{OptimizationFunction{true, Optimization.AutoZygote, NeuralPDE.var"#full_loss_function#308"{NeuralPDE.var"#null_nonadaptive_loss#115", Vector{NeuralPDE.var"#74#75"{NeuralPDE.var"#209#210"{RuntimeGeneratedFunctions.RuntimeGeneratedFunction{(:cord, Symbol("##θ#293"), :phi, :derivative, :integral, :u, :p), NeuralPDE.var"#_RGF_ModTag", NeuralPDE.var"#_RGF_ModTag", (0x03d798f9, 0x34fa0c63, 0xe7b60143, 0x8b2612da, 0x6b36183f), Expr}, NeuralPDE.var"#12#13", NeuralPDE.var"#269#276"{NeuralPDE.var"#269#270#277"{typeof(NeuralPDE.numeric_derivative)}, Dict{Symbol, Int64}, Dict{Symbol, Int64}, GridTraining{Float64}}, typeof(NeuralPDE.numeric_derivative), NeuralPDE.Phi{Optimisers.Restructure{Flux.Chain{Tuple{Flux.Dense{typeof(σ), Matrix{Float64}, Vector{Float64}}, Flux.Dense{typeof(σ), Matrix{Float64}, Vector{Float64}}, Flux.Dense{typeof(identity), Matrix{Float64}, Vector{Float64}}}}, NamedTuple{(:layers,), Tuple{Tuple{NamedTuple{(:weight, :bias, :σ), Tuple{Int64, Int64, Tuple{}}}, NamedTuple{(:weight, :bias, :σ), Tuple{Int64, Int64, Tuple{}}}, NamedTuple{(:weight, :bias, :σ), Tuple{Int64, Int64, Tuple{}}}}}}}, Nothing}, Vector{Float64}}, Matrix{Float64}}}, Vector{NeuralPDE.var"#74#75"{NeuralPDE.var"#209#210"{RuntimeGeneratedFunctions.RuntimeGeneratedFunction{(:cord, Symbol("##θ#293"), :phi, :derivative, :integral, :u, :p), NeuralPDE.var"#_RGF_ModTag", NeuralPDE.var"#_RGF_ModTag", (0x64128365, 0xe99c13c8, 0x3553718d, 0xc14f6af0, 0x258a1b45), Expr}, NeuralPDE.var"#12#13", NeuralPDE.var"#269#276"{NeuralPDE.var"#269#270#277"{typeof(NeuralPDE.numeric_derivative)}, Dict{Symbol, Int64}, Dict{Symbol, Int64}, GridTraining{Float64}}, typeof(NeuralPDE.numeric_derivative), NeuralPDE.Phi{Optimisers.Restructure{Flux.Chain{Tuple{Flux.Dense{typeof(σ), Matrix{Float64}, Vector{Float64}}, Flux.Dense{typeof(σ), Matrix{Float64}, Vector{Float64}}, Flux.Dense{typeof(identity), Matrix{Float64}, Vector{Float64}}}}, NamedTuple{(:layers,), Tuple{Tuple{NamedTuple{(:weight, :bias, :σ), Tuple{Int64, Int64, Tuple{}}}, NamedTuple{(:weight, :bias, :σ), Tuple{Int64, Int64, Tuple{}}}, NamedTuple{(:weight, :bias, :σ), Tuple{Int64, Int64, Tuple{}}}}}}}, Nothing}, Vector{Float64}}, Matrix{Float64}}}, NeuralPDE.PINNRepresentation, Bool, Vector{Int64}, Int64, NeuralPDE.Phi{Optimisers.Restructure{Flux.Chain{Tuple{Flux.Dense{typeof(σ), Matrix{Float64}, Vector{Float64}}, Flux.Dense{typeof(σ), Matrix{Float64}, Vector{Float64}}, Flux.Dense{typeof(identity), Matrix{Float64}, Vector{Float64}}}}, NamedTuple{(:layers,), Tuple{Tuple{NamedTuple{(:weight, :bias, :σ), Tuple{Int64, Int64, Tuple{}}}, NamedTuple{(:weight, :bias, :σ), Tuple{Int64, Int64, Tuple{}}}, NamedTuple{(:weight, :bias, :σ), Tuple{Int64, Int64, Tuple{}}}}}}}, Nothing}, typeof(additional_loss), Bool, Vector{Float64}}, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, typeof(SciMLBase.DEFAULT_OBSERVED_NO_TIME), Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing}, Optimization.ReInitCache{Vector{Float64}, SciMLBase.NullParameters}}}, Optimization.var"#231#240"{Optimization.var"#227#236"{OptimizationFunction{true, Optimization.AutoZygote, NeuralPDE.var"#full_loss_function#308"{NeuralPDE.var"#null_nonadaptive_loss#115", Vector{NeuralPDE.var"#74#75"{NeuralPDE.var"#209#210"{RuntimeGeneratedFunctions.RuntimeGeneratedFunction{(:cord, Symbol("##θ#293"), :phi, :derivative, :integral, :u, :p), NeuralPDE.var"#_RGF_ModTag", NeuralPDE.var"#_RGF_ModTag", (0x03d798f9, 0x34fa0c63, 0xe7b60143, 0x8b2612da, 0x6b36183f), Expr}, NeuralPDE.var"#12#13", NeuralPDE.var"#269#276"{NeuralPDE.var"#269#270#277"{typeof(NeuralPDE.numeric_derivative)}, Dict{Symbol, Int64}, Dict{Symbol, Int64}, GridTraining{Float64}}, typeof(NeuralPDE.numeric_derivative), NeuralPDE.Phi{Optimisers.Restructure{Flux.Chain{Tuple{Flux.Dense{typeof(σ), Matrix{Float64}, Vector{Float64}}, Flux.Dense{typeof(σ), Matrix{Float64}, Vector{Float64}}, Flux.Dense{typeof(identity), Matrix{Float64}, Vector{Float64}}}}, NamedTuple{(:layers,), Tuple{Tuple{NamedTuple{(:weight, :bias, :σ), Tuple{Int64, Int64, Tuple{}}}, NamedTuple{(:weight, :bias, :σ), Tuple{Int64, Int64, Tuple{}}}, NamedTuple{(:weight, :bias, :σ), Tuple{Int64, Int64, Tuple{}}}}}}}, Nothing}, Vector{Float64}}, Matrix{Float64}}}, Vector{NeuralPDE.var"#74#75"{NeuralPDE.var"#209#210"{RuntimeGeneratedFunctions.RuntimeGeneratedFunction{(:cord, Symbol("##θ#293"), :phi, :derivative, :integral, :u, :p), NeuralPDE.var"#_RGF_ModTag", NeuralPDE.var"#_RGF_ModTag", (0x64128365, 0xe99c13c8, 0x3553718d, 0xc14f6af0, 0x258a1b45), Expr}, NeuralPDE.var"#12#13", NeuralPDE.var"#269#276"{NeuralPDE.var"#269#270#277"{typeof(NeuralPDE.numeric_derivative)}, Dict{Symbol, Int64}, Dict{Symbol, Int64}, GridTraining{Float64}}, typeof(NeuralPDE.numeric_derivative), NeuralPDE.Phi{Optimisers.Restructure{Flux.Chain{Tuple{Flux.Dense{typeof(σ), Matrix{Float64}, Vector{Float64}}, Flux.Dense{typeof(σ), Matrix{Float64}, Vector{Float64}}, Flux.Dense{typeof(identity), Matrix{Float64}, Vector{Float64}}}}, NamedTuple{(:layers,), Tuple{Tuple{NamedTuple{(:weight, :bias, :σ), Tuple{Int64, Int64, Tuple{}}}, NamedTuple{(:weight, :bias, :σ), Tuple{Int64, Int64, Tuple{}}}, NamedTuple{(:weight, :bias, :σ), Tuple{Int64, Int64, Tuple{}}}}}}}, Nothing}, Vector{Float64}}, Matrix{Float64}}}, NeuralPDE.PINNRepresentation, Bool, Vector{Int64}, Int64, NeuralPDE.Phi{Optimisers.Restructure{Flux.Chain{Tuple{Flux.Dense{typeof(σ), Matrix{Float64}, Vector{Float64}}, Flux.Dense{typeof(σ), Matrix{Float64}, Vector{Float64}}, Flux.Dense{typeof(identity), Matrix{Float64}, Vector{Float64}}}}, NamedTuple{(:layers,), Tuple{Tuple{NamedTuple{(:weight, :bias, :σ), Tuple{Int64, Int64, Tuple{}}}, NamedTuple{(:weight, :bias, :σ), Tuple{Int64, Int64, Tuple{}}}, NamedTuple{(:weight, :bias, :σ), Tuple{Int64, Int64, Tuple{}}}}}}}, Nothing}, typeof(additional_loss), Bool, Vector{Float64}}, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, typeof(SciMLBase.DEFAULT_OBSERVED_NO_TIME), Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing}, Optimization.ReInitCache{Vector{Float64}, SciMLBase.NullParameters}}}, Optimization.var"#235#244", Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, typeof(SciMLBase.DEFAULT_OBSERVED_NO_TIME), Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing}, Optimization.ReInitCache{Vector{Float64}, SciMLBase.NullParameters}, Nothing, Nothing, Nothing, Nothing, Nothing, BFGS{LineSearches.InitialStatic{Float64}, LineSearches.HagerZhang{Float64, Base.RefValue{Bool}}, Nothing, Nothing, Flat}, Base.Iterators.Cycle{Tuple{Optimization.NullData}}, Bool, OptimizationOptimJL.var"#6#8"}}})
    @ Optim C:\Users\loren\.julia\packages\Optim\V8ZEC\src\multivariate\optimize\optimize.jl:36
 [64] __solve(cache::OptimizationOptimJL.OptimJLOptimizationCache{OptimizationFunction{false, Optimization.AutoZygote, OptimizationFunction{true, Optimization.AutoZygote, NeuralPDE.var"#full_loss_function#308"{NeuralPDE.var"#null_nonadaptive_loss#115", Vector{NeuralPDE.var"#74#75"{NeuralPDE.var"#209#210"{RuntimeGeneratedFunctions.RuntimeGeneratedFunction{(:cord, Symbol("##θ#293"), :phi, :derivative, :integral, :u, :p), NeuralPDE.var"#_RGF_ModTag", NeuralPDE.var"#_RGF_ModTag", (0x03d798f9, 0x34fa0c63, 0xe7b60143, 0x8b2612da, 0x6b36183f), Expr}, NeuralPDE.var"#12#13", NeuralPDE.var"#269#276"{NeuralPDE.var"#269#270#277"{typeof(NeuralPDE.numeric_derivative)}, Dict{Symbol, Int64}, Dict{Symbol, Int64}, GridTraining{Float64}}, typeof(NeuralPDE.numeric_derivative), NeuralPDE.Phi{Optimisers.Restructure{Flux.Chain{Tuple{Flux.Dense{typeof(σ), Matrix{Float64}, Vector{Float64}}, Flux.Dense{typeof(σ), Matrix{Float64}, Vector{Float64}}, Flux.Dense{typeof(identity), Matrix{Float64}, Vector{Float64}}}}, NamedTuple{(:layers,), Tuple{Tuple{NamedTuple{(:weight, :bias, :σ), Tuple{Int64, Int64, Tuple{}}}, NamedTuple{(:weight, :bias, :σ), Tuple{Int64, Int64, Tuple{}}}, NamedTuple{(:weight, :bias, :σ), Tuple{Int64, Int64, Tuple{}}}}}}}, Nothing}, Vector{Float64}}, Matrix{Float64}}}, Vector{NeuralPDE.var"#74#75"{NeuralPDE.var"#209#210"{RuntimeGeneratedFunctions.RuntimeGeneratedFunction{(:cord, Symbol("##θ#293"), :phi, :derivative, :integral, :u, :p), NeuralPDE.var"#_RGF_ModTag", NeuralPDE.var"#_RGF_ModTag", (0x64128365, 0xe99c13c8, 0x3553718d, 0xc14f6af0, 0x258a1b45), Expr}, NeuralPDE.var"#12#13", NeuralPDE.var"#269#276"{NeuralPDE.var"#269#270#277"{typeof(NeuralPDE.numeric_derivative)}, Dict{Symbol, Int64}, Dict{Symbol, Int64}, GridTraining{Float64}}, typeof(NeuralPDE.numeric_derivative), NeuralPDE.Phi{Optimisers.Restructure{Flux.Chain{Tuple{Flux.Dense{typeof(σ), Matrix{Float64}, Vector{Float64}}, Flux.Dense{typeof(σ), Matrix{Float64}, Vector{Float64}}, Flux.Dense{typeof(identity), Matrix{Float64}, Vector{Float64}}}}, NamedTuple{(:layers,), Tuple{Tuple{NamedTuple{(:weight, :bias, :σ), Tuple{Int64, Int64, Tuple{}}}, NamedTuple{(:weight, :bias, :σ), Tuple{Int64, Int64, Tuple{}}}, NamedTuple{(:weight, :bias, :σ), Tuple{Int64, Int64, Tuple{}}}}}}}, Nothing}, Vector{Float64}}, Matrix{Float64}}}, NeuralPDE.PINNRepresentation, Bool, Vector{Int64}, Int64, NeuralPDE.Phi{Optimisers.Restructure{Flux.Chain{Tuple{Flux.Dense{typeof(σ), Matrix{Float64}, Vector{Float64}}, Flux.Dense{typeof(σ), Matrix{Float64}, Vector{Float64}}, Flux.Dense{typeof(identity), Matrix{Float64}, Vector{Float64}}}}, NamedTuple{(:layers,), Tuple{Tuple{NamedTuple{(:weight, :bias, :σ), Tuple{Int64, Int64, Tuple{}}}, NamedTuple{(:weight, :bias, :σ), Tuple{Int64, Int64, Tuple{}}}, NamedTuple{(:weight, :bias, :σ), Tuple{Int64, Int64, Tuple{}}}}}}}, Nothing}, typeof(additional_loss), Bool, Vector{Float64}}, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, typeof(SciMLBase.DEFAULT_OBSERVED_NO_TIME), Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing}, Optimization.var"#228#237"{Optimization.var"#227#236"{OptimizationFunction{true, Optimization.AutoZygote, NeuralPDE.var"#full_loss_function#308"{NeuralPDE.var"#null_nonadaptive_loss#115", Vector{NeuralPDE.var"#74#75"{NeuralPDE.var"#209#210"{RuntimeGeneratedFunctions.RuntimeGeneratedFunction{(:cord, Symbol("##θ#293"), :phi, :derivative, :integral, :u, :p), NeuralPDE.var"#_RGF_ModTag", NeuralPDE.var"#_RGF_ModTag", (0x03d798f9, 0x34fa0c63, 0xe7b60143, 0x8b2612da, 0x6b36183f), Expr}, NeuralPDE.var"#12#13", NeuralPDE.var"#269#276"{NeuralPDE.var"#269#270#277"{typeof(NeuralPDE.numeric_derivative)}, Dict{Symbol, Int64}, Dict{Symbol, Int64}, GridTraining{Float64}}, typeof(NeuralPDE.numeric_derivative), NeuralPDE.Phi{Optimisers.Restructure{Flux.Chain{Tuple{Flux.Dense{typeof(σ), Matrix{Float64}, Vector{Float64}}, Flux.Dense{typeof(σ), Matrix{Float64}, Vector{Float64}}, Flux.Dense{typeof(identity), Matrix{Float64}, Vector{Float64}}}}, NamedTuple{(:layers,), Tuple{Tuple{NamedTuple{(:weight, :bias, :σ), Tuple{Int64, Int64, Tuple{}}}, NamedTuple{(:weight, :bias, :σ), Tuple{Int64, Int64, Tuple{}}}, NamedTuple{(:weight, :bias, :σ), Tuple{Int64, Int64, Tuple{}}}}}}}, Nothing}, Vector{Float64}}, Matrix{Float64}}}, Vector{NeuralPDE.var"#74#75"{NeuralPDE.var"#209#210"{RuntimeGeneratedFunctions.RuntimeGeneratedFunction{(:cord, Symbol("##θ#293"), :phi, :derivative, :integral, :u, :p), NeuralPDE.var"#_RGF_ModTag", NeuralPDE.var"#_RGF_ModTag", (0x64128365, 0xe99c13c8, 0x3553718d, 0xc14f6af0, 0x258a1b45), Expr}, NeuralPDE.var"#12#13", NeuralPDE.var"#269#276"{NeuralPDE.var"#269#270#277"{typeof(NeuralPDE.numeric_derivative)}, Dict{Symbol, Int64}, Dict{Symbol, Int64}, GridTraining{Float64}}, typeof(NeuralPDE.numeric_derivative), NeuralPDE.Phi{Optimisers.Restructure{Flux.Chain{Tuple{Flux.Dense{typeof(σ), Matrix{Float64}, Vector{Float64}}, Flux.Dense{typeof(σ), Matrix{Float64}, Vector{Float64}}, Flux.Dense{typeof(identity), Matrix{Float64}, Vector{Float64}}}}, NamedTuple{(:layers,), Tuple{Tuple{NamedTuple{(:weight, :bias, :σ), Tuple{Int64, Int64, Tuple{}}}, NamedTuple{(:weight, :bias, :σ), Tuple{Int64, Int64, Tuple{}}}, NamedTuple{(:weight, :bias, :σ), Tuple{Int64, Int64, Tuple{}}}}}}}, Nothing}, Vector{Float64}}, Matrix{Float64}}}, NeuralPDE.PINNRepresentation, Bool, Vector{Int64}, Int64, NeuralPDE.Phi{Optimisers.Restructure{Flux.Chain{Tuple{Flux.Dense{typeof(σ), Matrix{Float64}, Vector{Float64}}, Flux.Dense{typeof(σ), Matrix{Float64}, Vector{Float64}}, Flux.Dense{typeof(identity), Matrix{Float64}, Vector{Float64}}}}, NamedTuple{(:layers,), Tuple{Tuple{NamedTuple{(:weight, :bias, :σ), Tuple{Int64, Int64, Tuple{}}}, NamedTuple{(:weight, :bias, :σ), Tuple{Int64, Int64, Tuple{}}}, NamedTuple{(:weight, :bias, :σ), Tuple{Int64, Int64, Tuple{}}}}}}}, Nothing}, typeof(additional_loss), Bool, Vector{Float64}}, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, typeof(SciMLBase.DEFAULT_OBSERVED_NO_TIME), Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing}, Optimization.ReInitCache{Vector{Float64}, SciMLBase.NullParameters}}}, Optimization.var"#231#240"{Optimization.var"#227#236"{OptimizationFunction{true, Optimization.AutoZygote, NeuralPDE.var"#full_loss_function#308"{NeuralPDE.var"#null_nonadaptive_loss#115", Vector{NeuralPDE.var"#74#75"{NeuralPDE.var"#209#210"{RuntimeGeneratedFunctions.RuntimeGeneratedFunction{(:cord, Symbol("##θ#293"), :phi, :derivative, :integral, :u, :p), NeuralPDE.var"#_RGF_ModTag", NeuralPDE.var"#_RGF_ModTag", (0x03d798f9, 0x34fa0c63, 0xe7b60143, 0x8b2612da, 0x6b36183f), Expr}, NeuralPDE.var"#12#13", NeuralPDE.var"#269#276"{NeuralPDE.var"#269#270#277"{typeof(NeuralPDE.numeric_derivative)}, Dict{Symbol, Int64}, Dict{Symbol, Int64}, GridTraining{Float64}}, typeof(NeuralPDE.numeric_derivative), NeuralPDE.Phi{Optimisers.Restructure{Flux.Chain{Tuple{Flux.Dense{typeof(σ), Matrix{Float64}, Vector{Float64}}, Flux.Dense{typeof(σ), Matrix{Float64}, Vector{Float64}}, Flux.Dense{typeof(identity), Matrix{Float64}, Vector{Float64}}}}, NamedTuple{(:layers,), Tuple{Tuple{NamedTuple{(:weight, :bias, :σ), Tuple{Int64, Int64, Tuple{}}}, NamedTuple{(:weight, :bias, :σ), Tuple{Int64, Int64, Tuple{}}}, NamedTuple{(:weight, :bias, :σ), Tuple{Int64, Int64, Tuple{}}}}}}}, Nothing}, Vector{Float64}}, Matrix{Float64}}}, Vector{NeuralPDE.var"#74#75"{NeuralPDE.var"#209#210"{RuntimeGeneratedFunctions.RuntimeGeneratedFunction{(:cord, Symbol("##θ#293"), :phi, :derivative, :integral, :u, :p), NeuralPDE.var"#_RGF_ModTag", NeuralPDE.var"#_RGF_ModTag", (0x64128365, 0xe99c13c8, 0x3553718d, 0xc14f6af0, 0x258a1b45), Expr}, NeuralPDE.var"#12#13", NeuralPDE.var"#269#276"{NeuralPDE.var"#269#270#277"{typeof(NeuralPDE.numeric_derivative)}, Dict{Symbol, Int64}, Dict{Symbol, Int64}, GridTraining{Float64}}, typeof(NeuralPDE.numeric_derivative), NeuralPDE.Phi{Optimisers.Restructure{Flux.Chain{Tuple{Flux.Dense{typeof(σ), Matrix{Float64}, Vector{Float64}}, Flux.Dense{typeof(σ), Matrix{Float64}, Vector{Float64}}, Flux.Dense{typeof(identity), Matrix{Float64}, Vector{Float64}}}}, NamedTuple{(:layers,), Tuple{Tuple{NamedTuple{(:weight, :bias, :σ), Tuple{Int64, Int64, Tuple{}}}, NamedTuple{(:weight, :bias, :σ), Tuple{Int64, Int64, Tuple{}}}, NamedTuple{(:weight, :bias, :σ), Tuple{Int64, Int64, Tuple{}}}}}}}, Nothing}, Vector{Float64}}, Matrix{Float64}}}, NeuralPDE.PINNRepresentation, Bool, Vector{Int64}, Int64, NeuralPDE.Phi{Optimisers.Restructure{Flux.Chain{Tuple{Flux.Dense{typeof(σ), Matrix{Float64}, Vector{Float64}}, Flux.Dense{typeof(σ), Matrix{Float64}, Vector{Float64}}, Flux.Dense{typeof(identity), Matrix{Float64}, Vector{Float64}}}}, NamedTuple{(:layers,), Tuple{Tuple{NamedTuple{(:weight, :bias, :σ), Tuple{Int64, Int64, Tuple{}}}, NamedTuple{(:weight, :bias, :σ), Tuple{Int64, Int64, Tuple{}}}, NamedTuple{(:weight, :bias, :σ), Tuple{Int64, Int64, Tuple{}}}}}}}, Nothing}, typeof(additional_loss), Bool, Vector{Float64}}, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, typeof(SciMLBase.DEFAULT_OBSERVED_NO_TIME), Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing}, Optimization.ReInitCache{Vector{Float64}, SciMLBase.NullParameters}}}, Optimization.var"#235#244", Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, typeof(SciMLBase.DEFAULT_OBSERVED_NO_TIME), Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing}, Optimization.ReInitCache{Vector{Float64}, SciMLBase.NullParameters}, Nothing, Nothing, Nothing, Nothing, Nothing, BFGS{LineSearches.InitialStatic{Float64}, LineSearches.HagerZhang{Float64, Base.RefValue{Bool}}, Nothing, Nothing, Flat}, Base.Iterators.Cycle{Tuple{Optimization.NullData}}, Bool, OptimizationOptimJL.var"#6#8"})
    @ OptimizationOptimJL C:\Users\loren\.julia\packages\OptimizationOptimJL\uRfW9\src\OptimizationOptimJL.jl:249
 [65] solve!(cache::OptimizationOptimJL.OptimJLOptimizationCache{OptimizationFunction{false, Optimization.AutoZygote, OptimizationFunction{true, Optimization.AutoZygote, NeuralPDE.var"#full_loss_function#308"{NeuralPDE.var"#null_nonadaptive_loss#115", Vector{NeuralPDE.var"#74#75"{NeuralPDE.var"#209#210"{RuntimeGeneratedFunctions.RuntimeGeneratedFunction{(:cord, Symbol("##θ#293"), :phi, :derivative, :integral, :u, :p), NeuralPDE.var"#_RGF_ModTag", NeuralPDE.var"#_RGF_ModTag", (0x03d798f9, 0x34fa0c63, 0xe7b60143, 0x8b2612da, 0x6b36183f), Expr}, NeuralPDE.var"#12#13", NeuralPDE.var"#269#276"{NeuralPDE.var"#269#270#277"{typeof(NeuralPDE.numeric_derivative)}, Dict{Symbol, Int64}, Dict{Symbol, Int64}, GridTraining{Float64}}, typeof(NeuralPDE.numeric_derivative), NeuralPDE.Phi{Optimisers.Restructure{Flux.Chain{Tuple{Flux.Dense{typeof(σ), Matrix{Float64}, Vector{Float64}}, Flux.Dense{typeof(σ), Matrix{Float64}, Vector{Float64}}, Flux.Dense{typeof(identity), Matrix{Float64}, Vector{Float64}}}}, NamedTuple{(:layers,), Tuple{Tuple{NamedTuple{(:weight, :bias, :σ), Tuple{Int64, Int64, Tuple{}}}, NamedTuple{(:weight, :bias, :σ), Tuple{Int64, Int64, Tuple{}}}, NamedTuple{(:weight, :bias, :σ), Tuple{Int64, Int64, Tuple{}}}}}}}, Nothing}, Vector{Float64}}, Matrix{Float64}}}, Vector{NeuralPDE.var"#74#75"{NeuralPDE.var"#209#210"{RuntimeGeneratedFunctions.RuntimeGeneratedFunction{(:cord, Symbol("##θ#293"), :phi, :derivative, :integral, :u, :p), NeuralPDE.var"#_RGF_ModTag", NeuralPDE.var"#_RGF_ModTag", (0x64128365, 0xe99c13c8, 0x3553718d, 0xc14f6af0, 0x258a1b45), Expr}, NeuralPDE.var"#12#13", NeuralPDE.var"#269#276"{NeuralPDE.var"#269#270#277"{typeof(NeuralPDE.numeric_derivative)}, Dict{Symbol, Int64}, Dict{Symbol, Int64}, GridTraining{Float64}}, typeof(NeuralPDE.numeric_derivative), NeuralPDE.Phi{Optimisers.Restructure{Flux.Chain{Tuple{Flux.Dense{typeof(σ), Matrix{Float64}, Vector{Float64}}, Flux.Dense{typeof(σ), Matrix{Float64}, Vector{Float64}}, Flux.Dense{typeof(identity), Matrix{Float64}, Vector{Float64}}}}, NamedTuple{(:layers,), Tuple{Tuple{NamedTuple{(:weight, :bias, :σ), Tuple{Int64, Int64, Tuple{}}}, NamedTuple{(:weight, :bias, :σ), Tuple{Int64, Int64, Tuple{}}}, NamedTuple{(:weight, :bias, :σ), Tuple{Int64, Int64, Tuple{}}}}}}}, Nothing}, Vector{Float64}}, Matrix{Float64}}}, NeuralPDE.PINNRepresentation, Bool, Vector{Int64}, Int64, NeuralPDE.Phi{Optimisers.Restructure{Flux.Chain{Tuple{Flux.Dense{typeof(σ), Matrix{Float64}, Vector{Float64}}, Flux.Dense{typeof(σ), Matrix{Float64}, Vector{Float64}}, Flux.Dense{typeof(identity), Matrix{Float64}, Vector{Float64}}}}, NamedTuple{(:layers,), Tuple{Tuple{NamedTuple{(:weight, :bias, :σ), Tuple{Int64, Int64, Tuple{}}}, NamedTuple{(:weight, :bias, :σ), Tuple{Int64, Int64, Tuple{}}}, NamedTuple{(:weight, :bias, :σ), Tuple{Int64, Int64, Tuple{}}}}}}}, Nothing}, typeof(additional_loss), Bool, Vector{Float64}}, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, typeof(SciMLBase.DEFAULT_OBSERVED_NO_TIME), Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing}, Optimization.var"#228#237"{Optimization.var"#227#236"{OptimizationFunction{true, Optimization.AutoZygote, NeuralPDE.var"#full_loss_function#308"{NeuralPDE.var"#null_nonadaptive_loss#115", Vector{NeuralPDE.var"#74#75"{NeuralPDE.var"#209#210"{RuntimeGeneratedFunctions.RuntimeGeneratedFunction{(:cord, Symbol("##θ#293"), :phi, :derivative, :integral, :u, :p), NeuralPDE.var"#_RGF_ModTag", NeuralPDE.var"#_RGF_ModTag", (0x03d798f9, 0x34fa0c63, 0xe7b60143, 0x8b2612da, 0x6b36183f), Expr}, NeuralPDE.var"#12#13", NeuralPDE.var"#269#276"{NeuralPDE.var"#269#270#277"{typeof(NeuralPDE.numeric_derivative)}, Dict{Symbol, Int64}, Dict{Symbol, Int64}, GridTraining{Float64}}, typeof(NeuralPDE.numeric_derivative), NeuralPDE.Phi{Optimisers.Restructure{Flux.Chain{Tuple{Flux.Dense{typeof(σ), Matrix{Float64}, Vector{Float64}}, Flux.Dense{typeof(σ), Matrix{Float64}, Vector{Float64}}, Flux.Dense{typeof(identity), Matrix{Float64}, Vector{Float64}}}}, NamedTuple{(:layers,), Tuple{Tuple{NamedTuple{(:weight, :bias, :σ), Tuple{Int64, Int64, Tuple{}}}, NamedTuple{(:weight, :bias, :σ), Tuple{Int64, Int64, Tuple{}}}, NamedTuple{(:weight, :bias, :σ), Tuple{Int64, Int64, Tuple{}}}}}}}, Nothing}, Vector{Float64}}, Matrix{Float64}}}, Vector{NeuralPDE.var"#74#75"{NeuralPDE.var"#209#210"{RuntimeGeneratedFunctions.RuntimeGeneratedFunction{(:cord, Symbol("##θ#293"), :phi, :derivative, :integral, :u, :p), NeuralPDE.var"#_RGF_ModTag", NeuralPDE.var"#_RGF_ModTag", (0x64128365, 0xe99c13c8, 0x3553718d, 0xc14f6af0, 0x258a1b45), Expr}, NeuralPDE.var"#12#13", NeuralPDE.var"#269#276"{NeuralPDE.var"#269#270#277"{typeof(NeuralPDE.numeric_derivative)}, Dict{Symbol, Int64}, Dict{Symbol, Int64}, GridTraining{Float64}}, typeof(NeuralPDE.numeric_derivative), NeuralPDE.Phi{Optimisers.Restructure{Flux.Chain{Tuple{Flux.Dense{typeof(σ), Matrix{Float64}, Vector{Float64}}, Flux.Dense{typeof(σ), Matrix{Float64}, Vector{Float64}}, Flux.Dense{typeof(identity), Matrix{Float64}, Vector{Float64}}}}, NamedTuple{(:layers,), Tuple{Tuple{NamedTuple{(:weight, :bias, :σ), Tuple{Int64, Int64, Tuple{}}}, NamedTuple{(:weight, :bias, :σ), Tuple{Int64, Int64, Tuple{}}}, NamedTuple{(:weight, :bias, :σ), Tuple{Int64, Int64, Tuple{}}}}}}}, Nothing}, Vector{Float64}}, Matrix{Float64}}}, NeuralPDE.PINNRepresentation, Bool, Vector{Int64}, Int64, NeuralPDE.Phi{Optimisers.Restructure{Flux.Chain{Tuple{Flux.Dense{typeof(σ), Matrix{Float64}, Vector{Float64}}, Flux.Dense{typeof(σ), Matrix{Float64}, Vector{Float64}}, Flux.Dense{typeof(identity), Matrix{Float64}, Vector{Float64}}}}, NamedTuple{(:layers,), Tuple{Tuple{NamedTuple{(:weight, :bias, :σ), Tuple{Int64, Int64, Tuple{}}}, NamedTuple{(:weight, :bias, :σ), Tuple{Int64, Int64, Tuple{}}}, NamedTuple{(:weight, :bias, :σ), Tuple{Int64, Int64, Tuple{}}}}}}}, Nothing}, typeof(additional_loss), Bool, Vector{Float64}}, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, typeof(SciMLBase.DEFAULT_OBSERVED_NO_TIME), Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing}, Optimization.ReInitCache{Vector{Float64}, SciMLBase.NullParameters}}}, Optimization.var"#231#240"{Optimization.var"#227#236"{OptimizationFunction{true, Optimization.AutoZygote, NeuralPDE.var"#full_loss_function#308"{NeuralPDE.var"#null_nonadaptive_loss#115", Vector{NeuralPDE.var"#74#75"{NeuralPDE.var"#209#210"{RuntimeGeneratedFunctions.RuntimeGeneratedFunction{(:cord, Symbol("##θ#293"), :phi, :derivative, :integral, :u, :p), NeuralPDE.var"#_RGF_ModTag", NeuralPDE.var"#_RGF_ModTag", (0x03d798f9, 0x34fa0c63, 0xe7b60143, 0x8b2612da, 0x6b36183f), Expr}, NeuralPDE.var"#12#13", NeuralPDE.var"#269#276"{NeuralPDE.var"#269#270#277"{typeof(NeuralPDE.numeric_derivative)}, Dict{Symbol, Int64}, Dict{Symbol, Int64}, GridTraining{Float64}}, typeof(NeuralPDE.numeric_derivative), NeuralPDE.Phi{Optimisers.Restructure{Flux.Chain{Tuple{Flux.Dense{typeof(σ), Matrix{Float64}, Vector{Float64}}, Flux.Dense{typeof(σ), Matrix{Float64}, Vector{Float64}}, Flux.Dense{typeof(identity), Matrix{Float64}, Vector{Float64}}}}, NamedTuple{(:layers,), Tuple{Tuple{NamedTuple{(:weight, :bias, :σ), Tuple{Int64, Int64, Tuple{}}}, NamedTuple{(:weight, :bias, :σ), Tuple{Int64, Int64, Tuple{}}}, NamedTuple{(:weight, :bias, :σ), Tuple{Int64, Int64, Tuple{}}}}}}}, Nothing}, Vector{Float64}}, Matrix{Float64}}}, Vector{NeuralPDE.var"#74#75"{NeuralPDE.var"#209#210"{RuntimeGeneratedFunctions.RuntimeGeneratedFunction{(:cord, Symbol("##θ#293"), :phi, :derivative, :integral, :u, :p), NeuralPDE.var"#_RGF_ModTag", NeuralPDE.var"#_RGF_ModTag", (0x64128365, 0xe99c13c8, 0x3553718d, 0xc14f6af0, 0x258a1b45), Expr}, NeuralPDE.var"#12#13", NeuralPDE.var"#269#276"{NeuralPDE.var"#269#270#277"{typeof(NeuralPDE.numeric_derivative)}, Dict{Symbol, Int64}, Dict{Symbol, Int64}, GridTraining{Float64}}, typeof(NeuralPDE.numeric_derivative), NeuralPDE.Phi{Optimisers.Restructure{Flux.Chain{Tuple{Flux.Dense{typeof(σ), Matrix{Float64}, Vector{Float64}}, Flux.Dense{typeof(σ), Matrix{Float64}, Vector{Float64}}, Flux.Dense{typeof(identity), Matrix{Float64}, Vector{Float64}}}}, NamedTuple{(:layers,), Tuple{Tuple{NamedTuple{(:weight, :bias, :σ), Tuple{Int64, Int64, Tuple{}}}, NamedTuple{(:weight, :bias, :σ), Tuple{Int64, Int64, Tuple{}}}, NamedTuple{(:weight, :bias, :σ), Tuple{Int64, Int64, Tuple{}}}}}}}, Nothing}, Vector{Float64}}, Matrix{Float64}}}, NeuralPDE.PINNRepresentation, Bool, Vector{Int64}, Int64, NeuralPDE.Phi{Optimisers.Restructure{Flux.Chain{Tuple{Flux.Dense{typeof(σ), Matrix{Float64}, Vector{Float64}}, Flux.Dense{typeof(σ), Matrix{Float64}, Vector{Float64}}, Flux.Dense{typeof(identity), Matrix{Float64}, Vector{Float64}}}}, NamedTuple{(:layers,), Tuple{Tuple{NamedTuple{(:weight, :bias, :σ), Tuple{Int64, Int64, Tuple{}}}, NamedTuple{(:weight, :bias, :σ), Tuple{Int64, Int64, Tuple{}}}, NamedTuple{(:weight, :bias, :σ), Tuple{Int64, Int64, Tuple{}}}}}}}, Nothing}, typeof(additional_loss), Bool, Vector{Float64}}, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, typeof(SciMLBase.DEFAULT_OBSERVED_NO_TIME), Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing}, Optimization.ReInitCache{Vector{Float64}, SciMLBase.NullParameters}}}, Optimization.var"#235#244", Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, typeof(SciMLBase.DEFAULT_OBSERVED_NO_TIME), Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing}, Optimization.ReInitCache{Vector{Float64}, SciMLBase.NullParameters}, Nothing, Nothing, Nothing, Nothing, Nothing, BFGS{LineSearches.InitialStatic{Float64}, LineSearches.HagerZhang{Float64, Base.RefValue{Bool}}, Nothing, Nothing, Flat}, Base.Iterators.Cycle{Tuple{Optimization.NullData}}, Bool, OptimizationOptimJL.var"#6#8"})
    @ SciMLBase C:\Users\loren\.julia\packages\SciMLBase\QqtZA\src\solve.jl:160
 [66] solve(::OptimizationProblem{true, OptimizationFunction{true, Optimization.AutoZygote, NeuralPDE.var"#full_loss_function#308"{NeuralPDE.var"#null_nonadaptive_loss#115", Vector{NeuralPDE.var"#74#75"{NeuralPDE.var"#209#210"{RuntimeGeneratedFunctions.RuntimeGeneratedFunction{(:cord, Symbol("##θ#293"), :phi, :derivative, :integral, :u, :p), NeuralPDE.var"#_RGF_ModTag", NeuralPDE.var"#_RGF_ModTag", (0x03d798f9, 0x34fa0c63, 0xe7b60143, 0x8b2612da, 0x6b36183f), Expr}, NeuralPDE.var"#12#13", NeuralPDE.var"#269#276"{NeuralPDE.var"#269#270#277"{typeof(NeuralPDE.numeric_derivative)}, Dict{Symbol, Int64}, Dict{Symbol, Int64}, GridTraining{Float64}}, typeof(NeuralPDE.numeric_derivative), NeuralPDE.Phi{Optimisers.Restructure{Flux.Chain{Tuple{Flux.Dense{typeof(σ), Matrix{Float64}, Vector{Float64}}, Flux.Dense{typeof(σ), Matrix{Float64}, Vector{Float64}}, Flux.Dense{typeof(identity), Matrix{Float64}, Vector{Float64}}}}, NamedTuple{(:layers,), Tuple{Tuple{NamedTuple{(:weight, :bias, :σ), Tuple{Int64, Int64, Tuple{}}}, NamedTuple{(:weight, :bias, :σ), Tuple{Int64, Int64, Tuple{}}}, NamedTuple{(:weight, :bias, :σ), Tuple{Int64, Int64, Tuple{}}}}}}}, Nothing}, Vector{Float64}}, Matrix{Float64}}}, Vector{NeuralPDE.var"#74#75"{NeuralPDE.var"#209#210"{RuntimeGeneratedFunctions.RuntimeGeneratedFunction{(:cord, Symbol("##θ#293"), :phi, :derivative, :integral, :u, :p), NeuralPDE.var"#_RGF_ModTag", NeuralPDE.var"#_RGF_ModTag", (0x64128365, 0xe99c13c8, 0x3553718d, 0xc14f6af0, 0x258a1b45), Expr}, NeuralPDE.var"#12#13", NeuralPDE.var"#269#276"{NeuralPDE.var"#269#270#277"{typeof(NeuralPDE.numeric_derivative)}, Dict{Symbol, Int64}, Dict{Symbol, Int64}, GridTraining{Float64}}, typeof(NeuralPDE.numeric_derivative), NeuralPDE.Phi{Optimisers.Restructure{Flux.Chain{Tuple{Flux.Dense{typeof(σ), Matrix{Float64}, Vector{Float64}}, Flux.Dense{typeof(σ), Matrix{Float64}, Vector{Float64}}, Flux.Dense{typeof(identity), Matrix{Float64}, Vector{Float64}}}}, NamedTuple{(:layers,), Tuple{Tuple{NamedTuple{(:weight, :bias, :σ), Tuple{Int64, Int64, Tuple{}}}, NamedTuple{(:weight, :bias, :σ), Tuple{Int64, Int64, Tuple{}}}, NamedTuple{(:weight, :bias, :σ), Tuple{Int64, Int64, Tuple{}}}}}}}, Nothing}, Vector{Float64}}, Matrix{Float64}}}, NeuralPDE.PINNRepresentation, Bool, Vector{Int64}, Int64, NeuralPDE.Phi{Optimisers.Restructure{Flux.Chain{Tuple{Flux.Dense{typeof(σ), Matrix{Float64}, Vector{Float64}}, Flux.Dense{typeof(σ), Matrix{Float64}, Vector{Float64}}, Flux.Dense{typeof(identity), Matrix{Float64}, Vector{Float64}}}}, NamedTuple{(:layers,), Tuple{Tuple{NamedTuple{(:weight, :bias, :σ), Tuple{Int64, Int64, Tuple{}}}, NamedTuple{(:weight, :bias, :σ), Tuple{Int64, Int64, Tuple{}}}, NamedTuple{(:weight, :bias, :σ), Tuple{Int64, Int64, Tuple{}}}}}}}, Nothing}, typeof(additional_loss), Bool, Vector{Float64}}, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, typeof(SciMLBase.DEFAULT_OBSERVED_NO_TIME), Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing}, Vector{Float64}, SciMLBase.NullParameters, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Base.Pairs{Symbol, Union{}, Tuple{}, NamedTuple{(), Tuple{}}}}, ::BFGS{LineSearches.InitialStatic{Float64}, LineSearches.HagerZhang{Float64, Base.RefValue{Bool}}, Nothing, Nothing, Flat}; kwargs::Base.Pairs{Symbol, Int64, Tuple{Symbol}, NamedTuple{(:maxiters,), Tuple{Int64}}})
    @ SciMLBase C:\Users\loren\.julia\packages\SciMLBase\QqtZA\src\solve.jl:81

Astitva Aggarwal · Answer 19 · Mon Nov 13 2023 20:44:44 GMT+0800 (China Standard Time)

okay, did some minor changes to your new code and it works but ive made T_in chain input as 1 dim as that only depends on t (time) in your variable definition also in the discretization function used [ ] for parameter estimation to work properly. also why is dt=600? your domain is in minutes anyways, also dataset points is for every 10 minutes , so you would want to break that domain down into a grid spacing of 10 mins and not 600 as 600 would imply 600 minutes here. also the code :

using NeuralPDE, Lux, ModelingToolkit, DataFrames, CSV, DataLoaders, Flux, IntervalSets,
    Optimization, OptimizationOptimJL

# Definisci il modello dell'equazione differenziale
@parameters t, R, C, Cs

@variables T_in(..),
T_ext(..),
Q_heating(..),
Q_cooling(..),
Q_sun(..),
Q_lights(..),
Q_equipment(..)

# R, C, Cs = [1, 2, 3]

Dt = Differential(t)

# eqs = Dt(T_in(t)) ~ (-T_ext(t) + T_in(t)) / (R * C)
eqs = Dt(T_in(t)) ~ (T_in(t) - T_ext(t)) / (R * C) + Q_heating(t) / C - Q_cooling(t) / C +
                    Q_sun(t) / Cs + (Q_lights(t) + Q_equipment(t)) / C

domains = [t ∈ Interval(0.0, 365.0 * 24.0 * 60.0)]
bcs = [Dt(T_in(0.0)) ~ 4.48]

dt = 600.0  # 600 seconds (10 minute)

# Define the temporal space
tspan = (0.0, 365.0 * 24.0 * 60.0)

# load sampled data from CSV
data = CSV.File("shoebox_free.csv") |> DataFrame

# Put the sampled data in dedicated variables
T_in_data = data."OSGB1000005735772_FLOOR_1:Zone Mean Air Temperature [C](TimeStep)"[1:500]
T_ext_data = data."Environment:Site Outdoor Air Drybulb Temperature [C](TimeStep)"[1:500]
Q_heating_data = data."OSGB1000005735772_FLOOR_1_HVAC:Zone Ideal Loads Zone Total Heating Rate [W](TimeStep)"[1:500]
Q_cooling_data = data."OSGB1000005735772_FLOOR_1_HVAC:Zone Ideal Loads Zone Total Cooling Rate [W](TimeStep)"[1:500]
Q_sun_data = data."Environment:Site Direct Solar Radiation Rate per Area [W/m2](TimeStep)"[1:500]
Q_lights_data = data."OSGB1000005735772_FLOOR_1:Zone Lights Total Heating Rate [W](TimeStep)"[1:500]
Q_equipment_data = data."OSGB1000005735772_FLOOR_1:Zone Electric Equipment Total Heating Rate [W](TimeStep)"[1:500]

t_data = collect(Float64, 1:size(data, 1))

dataloader = DataLoader([
    vcat(T_in_data,
        T_ext_data,
        Q_heating_data,
        Q_cooling_data,
        Q_sun_data,
        Q_lights_data,
        Q_equipment_data,
        t_data),
])

# Define the NN
input_dim = 1
hidden_units = 8
len = length(t_data)

chain1 = Flux.Chain(Flux.Dense(input_dim, hidden_units, σ),
    Flux.Dense(hidden_units, hidden_units, σ),
    Flux.Dense(hidden_units, 1)) |>
         Flux.f64

indvars = [t]
depvars = [:T_in, :T_ext, :Q_heating, :Q_cooling, :Q_sun, :Q_lights, :Q_equipment]
u_ = hcat(T_in_data,
    T_ext_data,
    Q_heating_data,
    Q_cooling_data,
    Q_sun_data,
    Q_lights_data,
    Q_equipment_data)

init_params = [Flux.destructure(chain1)[1]]
acum = [0; accumulate(+, length.(init_params))]
sep = [(acum[i] + 1):acum[i + 1] for i in 1:(length(acum) - 1)]

function additional_loss(phi, θ, p)
    return sum(sum(abs2, phi[1](t_data', θ[sep[1]]) .- u_[:, 1]))
end

discretization = NeuralPDE.PhysicsInformedNN([chain1],
    NeuralPDE.GridTraining(dt), param_estim = true, additional_loss = additional_loss)
prob = NeuralPDE.discretize(pde_system, discretization)

# callback
# Parameter Optimization
res = Optimization.solve(prob,
    BFGS(),
    maxiters = 1000, callback = callback)

LorenzoBottaccioli · Answer 20 · Tue Nov 14 2023 17:12:42 GMT+0800 (China Standard Time)

@AstitvaAggarwal if I runn your code I get firs an error that pde_ssytem is not defined and after defining it I get:

BoundsError: attempt to access 1-element Vector{UnitRange{Int64}} at index [2]

Stacktrace:
 [1] getindex(A::Vector{UnitRange{Int64}}, i1::Int64)
   @ Base .\essentials.jl:13
 [2] build_symbolic_loss_function(pinnrep::NeuralPDE.PINNRepresentation, eqs::Equation; eq_params::SciMLBase.NullParameters, param_estim::Bool, default_p::Nothing, bc_indvars::Vector{Any}, integrand::Nothing, dict_transformation_vars::Nothing, transformation_vars::Nothing, integrating_depvars::Vector{Symbol})
   @ NeuralPDE C:\Users\loren\.julia\packages\NeuralPDE\F4RlZ\src\discretize.jl:92
 [3] #290
   @ .\none:0 [inlined]
 [4] iterate
   @ .\generator.jl:47 [inlined]
 [5] collect(itr::Base.Generator{Base.Iterators.Zip{Tuple{Vector{Equation}, Vector{Vector{Any}}, Vector{Vector{Symbol}}}}, NeuralPDE.var"#290#304"{NeuralPDE.PINNRepresentation}})
   @ Base .\array.jl:782
 [6] symbolic_discretize(pde_system::PDESystem, discretization::PhysicsInformedNN{GridTraining{Float64}, Nothing, Vector{NeuralPDE.Phi{Optimisers.Restructure{Flux.Chain{Tuple{Flux.Dense{typeof(σ), Matrix{Float64}, Vector{Float64}}, Flux.Dense{typeof(σ), Matrix{Float64}, Vector{Float64}}, Flux.Dense{typeof(identity), Matrix{Float64}, Vector{Float64}}}}, NamedTuple{(:layers,), Tuple{Tuple{NamedTuple{(:weight, :bias, :σ), Tuple{Int64, Int64, Tuple{}}}, NamedTuple{(:weight, :bias, :σ), Tuple{Int64, Int64, Tuple{}}}, NamedTuple{(:weight, :bias, :σ), Tuple{Int64, Int64, Tuple{}}}}}}}, Nothing}}, typeof(NeuralPDE.numeric_derivative), Bool, typeof(additional_loss), Nothing, Nothing, Base.Pairs{Symbol, Union{}, Tuple{}, NamedTuple{(), Tuple{}}}})
   @ NeuralPDE C:\Users\loren\.julia\packages\NeuralPDE\F4RlZ\src\discretize.jl:542
 [7] discretize(pde_system::PDESystem, discretization::PhysicsInformedNN{GridTraining{Float64}, Nothing, Vector{NeuralPDE.Phi{Optimisers.Restructure{Flux.Chain{Tuple{Flux.Dense{typeof(σ), Matrix{Float64}, Vector{Float64}}, Flux.Dense{typeof(σ), Matrix{Float64}, Vector{Float64}}, Flux.Dense{typeof(identity), Matrix{Float64}, Vector{Float64}}}}, NamedTuple{(:layers,), Tuple{Tuple{NamedTuple{(:weight, :bias, :σ), Tuple{Int64, Int64, Tuple{}}}, NamedTuple{(:weight, :bias, :σ), Tuple{Int64, Int64, Tuple{}}}, NamedTuple{(:weight, :bias, :σ), Tuple{Int64, Int64, Tuple{}}}}}}}, Nothing}}, typeof(NeuralPDE.numeric_derivative), Bool, typeof(additional_loss), Nothing, Nothing, Base.Pairs{Symbol, Union{}, Tuple{}, NamedTuple{(), Tuple{}}}})
   @ NeuralPDE C:\Users\loren\.julia\packages\NeuralPDE\F4RlZ\src\discretize.jl:683
 [8] top-level scope
   @ In[2]:92

I use dt=600 because I thought that everything was in seconds so 600s=10min. Can you better explain how dt works? Why did you put chain1 to one input? I thought and wanted to use all the others as input because the affect T_in, some of the inputs are not controllable: T_ext (External Temperature), Q_sun (Solar radiation), but the others are controllable loads to the system and I want the network to give me outputs with respect to those values. There is still some things that I'm not getting, I have seen that you put all the data in u_ how do you tell to the network to be trained in u_ I see it only in the additional_loss how the equation in eqs uses this data to solve the ode problem? How dose it know in which position of the u_ are the data?

Sorry for all of this questions but coming from python those things for me in Julia remain unclear.

Astitva Aggarwal · Answer 21 · Thu Nov 16 2023 05:02:35 GMT+0800 (China Standard Time)

hey @LorenzoBottaccioli, ignore the previous code things got mixed up, check this out, `

using NeuralPDE, Lux, ModelingToolkit, DataFrames, CSV, DataLoaders, Flux, IntervalSets,
    Optimization, OptimizationOptimJL, Zygote

# Definisci il modello dell'equazione differenziale
@parameters t, R, C, Cs

@variables T_in(..)

Dt = Differential(t)

domains = [t ∈ Interval(0.0, 365.0 * 24.0 * 60.0)]
eqs = [T_in(t) ~ T_in(t)]
bcs = [Dt(T_in(0.0)) ~ 4.48]

dt = 10.0  # 600 seconds (10 minute)

# Define the temporal space
tspan = Interval(0.0, 365.0 * 24.0 * 60.0)

# load sampled data from CSV
data = CSV.File("shoebox_free.csv") |> DataFrame

# Put the sampled data in dedicated variables
T_in_data = data."OSGB1000005735772_FLOOR_1:Zone Mean Air Temperature [C](TimeStep)"[1:500]
T_ext_data = data."Environment:Site Outdoor Air Drybulb Temperature [C](TimeStep)"[1:500]
Q_heating_data = data."OSGB1000005735772_FLOOR_1_HVAC:Zone Ideal Loads Zone Total Heating Rate [W](TimeStep)"[1:500]
Q_cooling_data = data."OSGB1000005735772_FLOOR_1_HVAC:Zone Ideal Loads Zone Total Cooling Rate [W](TimeStep)"[1:500]
Q_sun_data = data."Environment:Site Direct Solar Radiation Rate per Area [W/m2](TimeStep)"[1:500]
Q_lights_data = data."OSGB1000005735772_FLOOR_1:Zone Lights Total Heating Rate [W](TimeStep)"[1:500]
Q_equipment_data = data."OSGB1000005735772_FLOOR_1:Zone Electric Equipment Total Heating Rate [W](TimeStep)"[1:500]

t_data = collect(Float64, 0.0:10.0:(365.0 * 24.0 * 60.0))[1:500]

dataloader = DataLoader([
    vcat(T_in_data,
        T_ext_data,
        Q_heating_data,
        Q_cooling_data,
        Q_sun_data,
        Q_lights_data,
        Q_equipment_data,
        t_data),
])

# Define the NN
input_dim = 1
hidden_units = 8
len = length(t_data)

chain1 = Flux.Chain(Flux.Dense(input_dim, hidden_units, σ),
    Flux.Dense(hidden_units, hidden_units, σ),
    Flux.Dense(hidden_units, 1))

init_params = [Flux.destructure(chain1)[1]]
acum = [0; accumulate(+, length.(init_params))]
sep = [(acum[i] + 1):acum[i + 1] for i in 1:(length(acum) - 1)]

Phi = NeuralPDE.Phi(chain1)
function additional_loss(phi, θ, p)
    tin = phi[1](t_data', θ[sep[1]])
    R, C, Cs = p

    phi_ = (p) -> Phi(p, θ[sep[1]])[1]
    dphi = Zygote.gradient(phi_, t_data')

    return sum(abs2, vec(tin[1, :]) .- T_in_data) + sum(sum(abs2,
        (tin' .- T_ext_data) ./ (R * C) +
        (Q_heating_data .- Q_cooling_data) ./ C +
        Q_sun_data ./ Cs +
        (Q_lights_data .+ Q_equipment_data) ./ C - dphi[1][1, :]))
end

@named pde_system = NeuralPDE.PDESystem(eqs, bcs,
    domains,
    [t],
    [
        T_in(t),
    ],
    [R, C, Cs],
    defaults = Dict([R => 1.0, C => 1.0, Cs => 1.0]))#[R, C, Cs])

discretization = NeuralPDE.PhysicsInformedNN([chain1],
    NeuralPDE.GridTraining([dt]), param_estim = true, additional_loss = additional_loss)
prob = NeuralPDE.discretize(pde_system, discretization)

callback = function (p, l)
    println("Current loss is: $l")
    return false
end

res = Optimization.solve(prob,
    BFGS(),
    maxiters = 1000, callback = callback)

` Also dt should be in same units as the timespan interval(in your case minutes), the actual eq you are trying to solve is in the additional_loss function whereas the eq passed into the pde_system is trivial (bc is original). I was thinking of using Interpolations.jl for your discrete data but that just makes things longer

Astitva Aggarwal · Answer 22 · Thu Nov 30 2023 23:18:03 GMT+0800 (China Standard Time)

@LorenzoBottaccioli does this solve your issue?

Sathvik Bhagavan · Answer 23 · Tue Feb 13 2024 20:59:47 GMT+0800 (China Standard Time)

Hey @LorenzoBottaccioli, now #795 is merged, you can do parameter estimation with NNODE. This is at the ODEProblem level.