Implements simple recurrent network and a stacked recurrent network in numpy and torch respectively. Both flavours implements a forward and backward function API that is resposible for handling the model behaviour in forward pass and backward pass. Backward pass has been implemented using native numpy/torch tensors and no autograd engines have been used to perform the backward pass.
main.py is a thin wrapper that calls the appropriate model class and trains a recurrent network on tinyshakespeare. After training for a while, we can autoregressively sample random poems from the model.
Models present in models/ can also be trained on linear algebra and linux source code.
- numpy
- pytorch
Edit the main.py file to configure a RNN model by specifying number of hidden layers, sequence_length and so on. Exceute the following command in terminal.
$ python3 main.py- Multilayer GRU and LSTM
- Transformer
MIT