Fairseq with transformer evolution
This is the codebase we use in the following paper, the code is based on fairseq:
- The Devil in Linear Transformer
- Toeplitz Neural Network for Sequence Modeling
- Hierarchically Gated Recurrent Neural Network for Sequence Modeling, NeurIPS 2023 spotlight
Install
pip install --editable ./
git clone https://github.com/Doraemonzzz/hgru-pytorch.git
pip install -e .