reservoirpy / awesome-reservoir-computing

Awesome tutorials, papers, projects and tools for Reservoir Computing techniques like Echo State Networks (ESN).

Awesome Reservoir Computing

Awesome tutorials, papers, projects and tools for Reservoir Computing techniques like Echo State Networks (ESN).

Table of contents

• Introduction
• Tutorials
• Papers
• Tools
• Contributing

Introduction

• A Practical Guide to Applying Echo State Networks (2012) by Mantas Lukosevicius. Complete guide about ESNs, from theory to implementation.
• Echo State Network on Scholarpedia, by Herbert Jaeger. Generic introduction to Reservoir Computing from Echo State Networks.

Tutorials

Tuning Hyperparameters

• Hinaut & Trouvain (2021) Which Hype for My New Task? Hints and Random Search for Echo State Networks Hyperparameters. In International Conference on Artificial Neural Networks (pp. 83-97).

Founders

Early Reservoirs

• Dominey (1995) Complex sensory-motor sequence learning based on recurrent state representation and reinforcement learning. Biol. Cybernetics, Vol. 73, 265-274

• Buonomano & Merzenich (1995) Temporal Information Transformed into a Spatial Code by a Neural Network with Realistic Properties. Science 267, 1028-1030

Reservoir 2000's

• Jaeger (2001) The "echo state" approach to analysing and training recurrent neural networks. GMD Report 148, GMD - German National Research Institute for Computer Science

• Maass et al. (2002) Real-time computing without stable states: A new framework for neural computation based on perturbations. Neural Computation, 14(11):2531-2560

Papers

Reviews

Theory of RC

Online learning

• Backpropagation-decorrelation: Steil, J. J. (2004). Backpropagation-decorrelation: Online recurrent learning with O(N) complexity. 2004 IEEE International Joint Conference on Neural Networks (IEEE Cat. No.04CH37541), 2, 843–848 vol.2.

• FORCE: Sussillo, D., & Abbott, L. F. (2009). Generating Coherent Patterns of Activity from Chaotic Neural Networks. Neuron, 63(4), 544–557.

• Reward-modulated Hebbian learning (3 factors rules): Hoerzer, G. M., Legenstein, R., & Maass, W. (2014). Emergence of Complex Computational Structures From Chaotic Neural Networks Through Reward-Modulated Hebbian Learning. Cerebral Cortex, 24(3), 677–690.

Intrinsic plasticity

• Schrauwen, B., Wardermann, M., Verstraeten, D., Steil, J. J., & Stroobandt, D. (2008). Improving reservoirs using intrinsic plasticity. Neurocomputing, 71(7), 1159–1171. https://doi.org/10.1016/j.neucom.2007.12.020

RC for neurosciences

• RC as model of prefrontal cortex activity: Enel, P., Procyk, E., Quilodran, R., & Dominey, P. F. (2016). Reservoir Computing Properties of Neural Dynamics in Prefrontal Cortex. PLOS Computational Biology, 12(6), e1004967.

• RC as model of the working memory: Strock, A., Hinaut, X., & Rougier, N. P. (2020). A Robust Model of Gated Working Memory. Neural Computation, 32(1), 153–181.

Recent papers

2021

• Pedrelli & Hinaut (2021). Hierarchical-task reservoir for online semantic analysis from continuous speech. IEEE Transactions on Neural Networks and Learning Systems.

• Manneschi et al. (2021). SpaRCe: Improved Learning of Reservoir Computing Systems through Sparse Representations. IEEE Transactions on Neural Networks and Learning Systems.

• Manneschi et al. (2021). Exploiting multiple timescales in hierarchical echo state networks. Frontiers in Applied Mathematics and Statistics, 6, 76.

2020

• Bianchi et al. (2020) Reservoir computing approaches for representation and classification of multivariate time series. IEEE transactions on neural networks and learning systems, 32(5), 2169-2179.

Tools

Contributing

Have anything in mind that you think is awesome and would fit in this list? Feel free to send a pull request.