This github respository is for our NeurIPS 2020 paper "Neural Networks Fail to Learn Periodic Functions and How to Fix It", see the arxiv version below: https://arxiv.org/abs/2006.08195
We have now added a Jupyter notebook file for studying the extrapolation and interpolation properties of neural networks. See 'extrapolation_experiments.ipynb'. In the notebook, a very elementary implementation of the Snake activation function is given.
- Extrapolation experiments and simple implementation of Snake.
- Human body temperature regression task
- Atmospheric temperature regression task
- Regression task on market index
- more advanced implementations of Snake (such as learnable a and initialization correction)
We are on our way cleaning our code! It should be ready within a week or two. We will let you know once we make an update.
In the mean-while, let me comment that Edward Dixon had made an implementation of our method a while ago. It can be found here: https://github.com/EdwardDixon/snake
Ziyin, Tilman