Pytorch implementations of:
- Transformer Encoder block from Attention Is All You Need.
- Evolved Transformer Encoder block from Evolved Transformer.
- Gated Linear Unit from Language Modelling with Gated Convolutional Networks.
for text classification on AG_NEWS dataset.
- numpy==1.17.2
- torch==1.2.0
- torchtext==0.5.0
- tqdm==4.44.1
- spacy==2.2.4
- Python 3.6+
Install prerequisites with:
pip3 install -r requirements.txt
python3 -m spacy download en
To run Encoder for text classification on AG_NEWS dataset:
python3 main.py
python3 main.py --evolved true
More configurations can be found here.
- Attention Is All You Need.
- Evolved Transformer.
- Language Modelling with Gated Convolutional Networks.
Shikhar / @Shikhar