Longformer-paddle Glossary Longformer: Long-Document Transformer LED: Longformer-Encoder-Decoder Related Works: Attention pattern: Sparse Transformer, BlockSparse. Longformer: RoBERTa. LED: Bart. Reference https://arxiv.org/pdf/2004.05150.pdf https://github.com/allenai/longformer https://github.com/huggingface/transformers/tree/master/src/transformers/models/longformer https://github.com/huggingface/transformers/tree/master/src/transformers/models/led longformer/longformer.py @LongformerSelfAttention