efficient-attention

There are 1 repository under efficient-attention topic.

lucidrains / ring-attention-pytorch
Implementation of 💍 Ring Attention, from Liu et al. at Berkeley AI, in Pytorch
attention-mechanism efficient-attention long-context distributed-attention
Language:Python 384
lucidrains / CoLT5-attention
Implementation of the conditionally routed attention in the CoLT5 architecture, in Pytorch
artificial-intelligence attention-mechanisms deep-learning efficient-attention routing
Language:Python 216
jlamprou / Infini-Attention
Efficient Infinite Context Transformers with Infini-attention Pytorch Implementation + QwenMoE Implementation + Training Script + 1M context keypass retrieval
attention efficient-attention infinite llm transformer qwen
Language:Python 44
davidsvy / cosformer-pytorch
Unofficial PyTorch implementation of the paper "cosFormer: Rethinking Softmax In Attention".
neural-network transformer pytorch attention-mechanism deep-learning artificial-intelligence efficient-attention iclr2022 iclr
Language:Jupyter Notebook 43
HolmesShuan / Compact-Global-Descriptor
Pytorch implementation of "Compact Global Descriptor for Neural Networks" (CGD).
attention-mechanism convolutional-neural-networks pytorch efficient efficient-attention attention-model
Language:Python 25
robflynnyh / hydra-linear-attention
Implementation of: Hydra Attention: Efficient Attention with Many Heads (https://arxiv.org/abs/2209.07484)
attention efficient-attention linear-attention machine-learning transformers
Language:Python 10
Ascend-Research / CascadedGaze
The official PyTorch implementation for CascadedGaze: Efficiency in Global Context Extraction for Image Restoration, TMLR'24.
deblurring denoising efficiency image-restoration transformer efficient-attention
Language:Python 5
gmlwns2000 / sea-attention
Official Implementation of SEA: Sparse Linear Attention with Estimated Attention Mask (ICLR 2024)
attention efficient-attention linear-attention sea-attention
Language:Python 4
MAGICS-LAB / NonparametricHopfield
Nonparametric Modern Hopfield Models
efficient-attention efficient-transformers modern-hopfield-model modern-hopfield-networks efficient-hopfield-models efficient-hopfield-networks
Language:Jupyter Notebook 1

efficient-attention

lucidrains / ring-attention-pytorch

lucidrains / CoLT5-attention

jlamprou / Infini-Attention

davidsvy / cosformer-pytorch

HolmesShuan / Compact-Global-Descriptor

robflynnyh / hydra-linear-attention

Ascend-Research / CascadedGaze

gmlwns2000 / sea-attention

MAGICS-LAB / NonparametricHopfield