#The code is largely based on [1], [2] and [3]
-Files:
---FS_example.py: Minimal example (20 lines of code) of how to create
a FS-LSTM and run one step of it, using the Tensorflow LSTM.
---main.py: Main file which should be run to replicate the PTB
and enwik8 experiments.
To run PTB experiment: python3 main.py --model ptb
To run enwik8 experiment: python3 main.py --model enwik8
We only include the PTB dataset due to size limitations.
To run the enwik8 experiments the dataset should be
added to "data/enwik8" divided in three files: "train",
"valid" and "test"
---config.py: Hyper-parameters are loaded from here according
to which experiment is being performed. All hyper-
parameters from Table 3 of the paper can be set
using this file.
---reader.py: Reads training, validation and testing data from
the data folder.
---LNLSTM.py: Layer normalized LSTM with Zoneout.
---aux.py: File with auxiliary functions for LNLSTM.py.
---FSRNN.py: Fast-Slow RNN class that can be used with any
standard Tensorflow RNN.
[1] https://github.com/tensorflow/models/tree/master/tutorials/rnn/ptb
[2] https://github.com/pbhatia243/tf-layer-norm
[3] https://github.com/hardmaru/supercell