linear-attention

There are 0 repository under linear-attention topic.

BlinkDL / RWKV-LM
RWKV is an RNN with transformer-level LLM performance. It can be directly trained like a GPT (parallelizable). So it's combining the best of RNN and transformer - great performance, fast inference, saves VRAM, fast training, "infinite" ctx_len, and free sentence embedding.
attention-mechanism deep-learning gpt gpt-2 gpt-3 language-model linear-attention lstm pytorch rnn transformer transformers rwkv chatgpt
Language:Python 11693
lucidrains / taylor-series-linear-attention
Explorations into the recently proposed Taylor Series Linear Attention
artificial-intelligence attention-mechanisms deep-learning linear-attention
Language:Python 78
lucidrains / agent-attention-pytorch
Implementation of Agent Attention in Pytorch
artificial-intelligence attention-mechanisms deep-learning linear-attention
Language:Python 73
lironui / Multi-Attention-Network
The semantic segmentation of remote sensing images
attention-mechanism linear-attention remote-sensing segmentation semantic-segmentation
Language:Python 54
lucidrains / autoregressive-linear-attention-cuda
CUDA implementation of autoregressive linear attention, with all the latest research findings
artificial-intelligence attention-mechanisms cuda deep-learning linear-attention
Language:Python 43
lironui / MAResU-Net
The semantic segmentation of remote sensing images
attention attention-mechanism linear-attention remote-sensing segmentation semantic-segmentation
Language:Python 41
robflynnyh / hydra-linear-attention
Implementation of: Hydra Attention: Efficient Attention with Many Heads (https://arxiv.org/abs/2209.07484)
attention efficient-attention linear-attention machine-learning transformers
Language:Python 10
RWKV-Wiki / rwkv-wiki.github.io
RWKV Wiki website (archived, please visit official wiki)
attention-mechanism deep-learning gpt gpt-2 gpt-3 language-model linear-attention lstm rnn rwkv transformer transformers
Language:Shell 9
mtanghu / LEAP
LEAP: Linear Explainable Attention in Parallel for causal language modeling with O(1) path length, and O(1) inference
additive-attention attention-mechanism deep-learning dot-product-attention linear-attention local-attention parallel pytorch rnn softmax transformer transformers
Language:Jupyter Notebook 4
gmlwns2000 / sea-attention
Official Implementation of SEA: Sparse Linear Attention with Estimated Attention Mask (ICLR 2024)
attention efficient-attention linear-attention sea-attention
Language:Python 3
Rushi314 / Transformers-for-high-resolution-image-synthesis
Taming Transformers for High-Resolution Image Synthesis
computer-vision gans linear-attention sflickr transformers-gpt2
Language:Jupyter Notebook 0
fattorib / flashy_linear_attention
Flash linear attention kernels in Triton
attention gpu transformer triton linear-attention
Language:Python

linear-attention

BlinkDL / RWKV-LM

lucidrains / taylor-series-linear-attention

lucidrains / agent-attention-pytorch

lironui / Multi-Attention-Network

lucidrains / autoregressive-linear-attention-cuda

lironui / MAResU-Net

robflynnyh / hydra-linear-attention

RWKV-Wiki / rwkv-wiki.github.io

mtanghu / LEAP

gmlwns2000 / sea-attention

Rushi314 / Transformers-for-high-resolution-image-synthesis

fattorib / flashy_linear_attention