Merge with JuliaPhysics or create JuliaDynamics?
Datseris opened this issue · comments
With the latest release of DynamicalSystems I think it is about time that this package as well DynamicalBilliards go into an organization. I just learned that there is an organization called JuliaPhysics. I personally feel like this general name actually suit these packages, because they are also supposed to be general, even though they are specialized. Plus, you don't want to flood the "market" with organizations.
I would like to ask others that have contributed here or at the DynamicalBilliards, how they feel like about it. I am tagging you, please let me know what you think.
@RainerEngelken
@SebastianM-C
@dapias
@ChrisRackauckas
@dpsanders
@dapias
Hi, I've just discovered DynamicalSystems.jl, DifferentialEquations.jl and all the Julia goodness, so I probably don't know what I'm talking about. I also don't know what the purpose and benefits of an organization are. Please excuse if I'm speaking nonsense.
It seems to me that what all these projects have in common is modelling and simulation, which could include domains outside of physics, like economics or information systems (though, depending on your view, you could also say that physics is everything 😄). What about JuliaModeling/JuliaModelling (US/UK spelling) or JuliaM&S (if ampersands are appropriate or allowed).
I'd also like to point out two projects in the modelling and simulation category that don't seem to have gotten that much attention: Sims.jl by @tshort and Modia.jl by Hilding Elmqvist, @MartinOtter and @toivoh.
I think the Julia ecosystem may benefit from a modular approach for dynamical systems.
It would be great if JuliaDynamics was trying to define abstract DynamicalSystems types (just like DiscreteDynamicalSystem and ContinuousDynamicalSystem) and an API so that one could implement algorithm on abstract type that would work on systems that were modeled by any of the modelling interfaces. This modular approach is explained in @ChrisRackauckas blog post here.
@raulrpearson mentionned two packages working with dynamical systems but there are plenty more. Some providing modeling, some providing simulation, some providing algorithm to compute other things such as checking stability. Having a common interface would improve their interoperability.
DynamicalSystems already does this but it does not cover all use cases. For instance, Sims.jl uses differential algebraic equations, some other packages (like ControlSystems work with control, some have stochasticity, some have perturbation, ...
I don't know what the interface covering all these cases would look like but one package (maybe called Systems.jl ?) defining abstract types for these and high level API that would live in a JuliaDynamics (or JuliaSystems ?) organisation would certainly be beneficial. The API would of course be extended for special cases of systems, the main package does not need to cover everything.
It is probably slightly off topic but I thought that it was the time to raise the question since you plan to create the JuliaDynamics organisation and I wanted to know what you think about it.
Organizations are about putting together common codebases so that way it's easy to manage permissions and perform org-wide reviews and maintenance. If you wouldn't expect a contributor of one to be a contributor to other pieces of code in the org than it probably doesn't fit into a single org (though you can have orgs for collecting codes I guess).
In that sense, I think it's hard to group together different simulation tools. So while
It seems to me that what all these projects have in common is modelling and simulation, which could include domains outside of physics, like economics or information systems
may somewhat apply to JuliaDiffEq (we do have DiffEqPhysics, DiffEqFinancial, DiffEqBiological, and MultiScaleArrays as modeling tools), I don't think it's quite necessary. Those repos are in JuliaDiffEq because I developed them and other people in JuliaDiffEq are interested in those projects and have helped as well. However, I think large modeling projects, even if they rely on DiffEq, will need their own org. The main focus of JuliaDiffEq is the development of the solver mechanisms with an interface that deep modeling tools can be built upon.
Not only that but trying to contribute to all of the different domains is just hard if not impossible. People have specialties and will only be working on economics or physics, so bringing the repos together only seems like it would cause disunity (in DiffEq it works only because the modeling tools are simple and because this is for testing solving models of different domains).
Creating a common interface between different modeling tools is difficult as well. Every domain seems to focus on different things. In systems biological models you want to talk about cells and concentrations, while in molecular dynamics models you want to talk about molecules. Economics is then talking about things which are completely different. The wording and the interface is all completely different. I'm not sure how much sharing these tools should or could have. The purpose of a common interface is to be able to build extension functionality that is shared, but what would daughter tracking even mean in economic models?
You can extend the API for economical or biological specific terms in different packages.
What would live in Systems.jl is probably have for example something like
abstract type AbstractSystem end
"""
dynamic(s::AbstractSystem, x:, ...)
Gives the value of the lefthand-side of the dynamical equations
"""
function dynamic end
"""
statejacobian(s::AbstractSystem, x, ...)
Gives the derivative of `dynamic` with respect to the state `x`.
"""
function statejacobian end
"""
statedimension(s::AbstractSystem, x, ...)
Gives the dimension of the state.
"""
function statedimension endFor a controlled system, you would use dynamic(s, x, u) and for an uncontrolled system you would use dynamic(s, x). You can add a trait or abstract type to see whether a system is controlled.
What do you plan on building that uses this directly and is cross-domain?
One example is HybridSystems which would extends Systems.jl to Hybrid Systems which is a pair of a classical system with a continuous state space and a discrete state space.
If an AbstractSystem type exist, I can just do
"""
HybridSystem{C, D}
Hybrid System with continuous state space system of type C and discrete state space system of type D
"""
struct HybridSystem{C, D}
c::C
d::D
endThe HybridSystems package could be used by user wanting to model hybrid systems and by user wanting to implement algorithm on hybrid system (e.g. SwitchTimeOpt or SwitchOnSafety).
On SwitchOnSafety for instance, it would be convenient to use dynamical dispatch to chose the appropriate method to compute the invariant set depending on the types C and D.
The user would for example use DynamicalSystems to create the continuous part and Automata to create the discrete part and then ask "is this stable" and SwitchOnSafety will be able to work on the problem without even needed to have the packages used to model the system in his REQUIRE file.
@Datseris Note that this may help with #14 since you will be able to detect on the type of C and D whether this is a subclass of Hybrid Systems that is deterministic.
@raulrpearson thanks for enlightening me, I have never seen before neither Sims.jl or ModiaSim. I am really, really glad that they do different things than my packages. Imagine how sad it would be if I found out that a package that does what I do already exists.
Organizations are about putting together common codebases so that way it's easy to manage permissions and perform org-wide reviews and maintenance. If you wouldn't expect a contributor of one to be a contributor to other pieces of code in the org than it probably doesn't fit into a single org (though you can have orgs for collecting codes I guess).
That is true that this is the reason github organizations were created. However, they also offer the very powerful opportunity of grouping similar things together. I don't expect the various packages of e.g. JuliaDynamics to contribute one to another, at least not significantly. But the point is, it is much easier to check what is inside an organization that is relevant to your field, than just randomly encountering packages. For example, I have seen what is inside JuliaDSP, JuliaQuantum, JuliaMath, JuliaStats (and some others as well), and I am aware of the packages in there. But, I have never encountered these aforementioned packages (e.g. Sims.jl) because I don't randomly search for packages but instead look at groups that feel interesting to me.
It is very likely that other people share this approach as well; grouping is helpful for a big set of individuals!
Also, in my opinion, over-grouping has to stop! It doesn't help anybody. There is absolutely no reason to create an organization that has only one package! It is better to compromise and have many packages in one organization, even though they don't share code or interfaces, but they share thematic content, than to have each one be separate. This is how I feel at least, that is why I will definitely put my packages to either the existing JuliaPhysics or to a new JuliaDynamics if somebody else wants to also join.
@blegat Many general interfaces are already defined in DynamicalSystems.jl, including the abstract type DynamicalSystem. For example of of the subtypes of that have defined dimension(), evolve() functions and other relevant stuff.
Of course, from my eyes, the point of this Issue is not to make a general interface for "Systems". That may, or may not be possible and may be fruitful if others also want to use it. What I really care about, is have some grouping umbrella that will help people identify things that have similar thematic content. Of course that is only possible if other package creators agree with this vision.
@raulrpearson have you used Sims.jl ? What different does it offer from DifferentialEquations.jl ? I could not find any difference from the examples, even Events are handled exellently by DiffEq.
I haven't had a chance to explore Sims.jl yet. I know about it and Modia.jl because I'm familiar with Modelica. My understanding is that both projects are attempts at recreating Modelica (or it's features) with the use of Julia. From Sims.jl documentation:
Sims is like a lite version of Modelica. This package is for non-causal modeling in Julia. The idea behind non-causal modeling is that the user develops models based on components which are described by a set of equations. A tool can then transform the equations and solve the differential algebraic equations. Non-causal models tend to match their physical counterparts in terms of their specification and implementation.
Hilding Elmqvist did a presentation in JuliaCon 2017 if you want to learn more about it.
I'm new to Julia and don't know that much about DifferentialEquations.jl, but I think the main difference compared with Sims.jl and Modia.jl is that these two have a bigger modelling component to them. They also have algorithms that deal with the resulting equations - including solvers - whereas DifferentialEquations.jl focuses mainly on solvers. From it's documentation:
This is a suite for numerically solving differential equations in Julia. The purpose of this package is to supply efficient Julia implementations of solvers for various differential equations.
My impression is that projects like Sims.jl and Modia.jl will probably make use of DifferentialEquations.jl so that they don't have to write their own solvers. For an engineer/scientist wanting to work on a specific problem, I think the first two provide a more convenient way to describe the system, that is, to specify a model of it. This is because it can be expressed in a non-causal component-based way.
@raulrpearson is correct. JuliaDiffEq is about solvers whereas these are modeling tools. Sims.jl is built on DASRK.jl and Sundials.jl from JuliaDiffEq. And uses its tools to build its own modeling interface. Modia.jl just uses (supposedly according to the README, though there doesn't exist code) Sundials' IDA. These modeling languages make it easy to build and compose systems. The electrical example
https://github.com/tshort/Sims.jl#electrical-example
is something that you can definitely implement in DiffEq directly, but it's easier to build and analyze a circuit with a proper modeling tool on top like this. These tend to be much easier to build complex models in, at the cost a bit more constrained in what they can do and what solvers they have access to (notice that all of these utilize only the fully implicit DAEs which are the least efficient but are the most general form of non-delay and non-stochastic ODE).
These are all really great tools and what I am trying to do with JuliaDiffEq is make everything performant and feature-rich enough that it can be a backend to these kinds of modeling tools (though of course, there will always be loads of people who want to handle the equations directly).
@giordano do you want to contribute something to the discussion?
I'm not sure this package completely fits in the physics domain because others have pointed out that this kind of modeling can be used outside physics.
Besides this concern, I would wholeheartedly welcome the package into JuliaPhysics if you think it's well within the physics domain.