Echo State Network code (used in parts of the DeepTurb project of Carl-Zeiss-Stiftung). Relies on a pytorch backend. Inspired by the easyesn package (https://github.com/kalekiu/easyesn).
Supports fully autonomous and teacher forced ESN predictions. A hybrid of both methods, called semi-techer forcing, is also implemented (restricted use).
In order to install the package just use pip3 install turbESN. Here you can find the current PyPi release https://pypi.org/project/turbESN/.
Stable for
python>= 3.6.0torch>= 1.10.0numpy>= 1.20.1h5py>= 2.10.0pyYAML>= 6.0
A first basic introduction can be found in the notbeookbasic_tour.ipynb. This example uses data from the Lorenz '63 system (https://en.wikipedia.org/wiki/Lorenz_system).
Classical Reservoir Computing (RC) makes use of a nonlinear dynamical expansion of the information to be processed. This information is lifted to a high-dimensional reservoir state. A linear readout after each reservoir update yields the reservoir output. The training scheme of RC is concerned with finding the right hyperplane of within the reservoir space, s.t. the reservoir output matches the expected output.Therefore only the linear readout is trained, usually to minimize the mean square error between output and target. Therefore the readout can be computed by a simple linear regression (often with Tikhonov regularization/ L2 penalty). This, always converging learning procedure, makes RC an extremely computationally efficient method. RC is more then just a method, it stands for new computing paradigm (in contrast to the Turing-von Neumann architecture).
The Echo State Network (ESN) is a certain implementation of RC. Its architecture follows a Recurrent Neural Network (RNN) structure, where only the output weights are trained by linear regression. This is also a simple way of circumventing the known exploding/ vanishing gradient problem which RNN suffer from.
-
F. Heyder and J. Schumacher - Phys. Rev. E 103, 053107 (2021): "Echo state network for two-dimensional turbulent moist Rayleigh-Bénard convection"
-
P. Pfeffer, F. Heyder and J. Schumacher - Phys Rev. Res. 4, 033176 (2022): "Hybrid quantum-classical reservoir computing of thermal convection flow"
-
F. Heyder and J. P. Mellado and J. Schumacher - Phys. Rev. E 106, 055303 (2022): "Generalizability of reservoir computing for flux-driven two-dimensional convection"
-
M. S. Ghazijahani, F. Heyder, J. Schumacher and C. Cierpka, Meas. Sci. Technol. 34 (2023) 014002 "On the benefits and limitations of Echo State Networks for turbulent flow prediction"
- M. Lukoševičius - Tricks of the Trade. Lecture Notes in Computer Science, Vol. 7700 (2012): "A Practical Guide to Applying Echo State Networks".
- M. Lukoševičius and A. Uselis - Cogn. Comput. (2021): "Efficient Implementations of Echo State Network Cross-Validation"
- X. Hinaut and N. Trouvain - (2021) Artificial Neural Networks and Machine Learning – ICANN 2021: "Which Hype for My New Task? Hints and Random Search for Echo State Networks Hyperparameter"