self-attention

There are 23 repositories under self-attention topic.

datawhalechina / leedl-tutorial
《李宏毅深度学习教程》（李宏毅老师推荐👍，苹果书🍎），PDF下载地址：https://github.com/datawhalechina/leedl-tutorial/releases
bert chatgpt cnn deep-learning diffusion gan leedl-tutorial machine-learning network-compression pruning reinforcement-learning rnn self-attention transfer-learning transformer tutorial
Language:Jupyter Notebook 15967
zhouhaoyi / Informer2020
The GitHub repository for the paper "Informer" accepted by AAAI 2021.
deep-learning forecasting pytorch self-attention time-series transformer
Language:Python 6288
cmhungsteve / Awesome-Transformer-Attention
An ultimately comprehensive paper list of Vision Transformer/Attention, including papers, codes, and related websites
attention-mechanism attention-mechanisms awesome-list computer-vision deep-learning detr papers self-attention transformer transformer-architecture transformer-awesome transformer-cv transformer-models transformer-with-cv transformers vision-transformer visual-transformer vit
4959
PetarV- / GAT
Graph Attention Networks (https://arxiv.org/abs/1710.10903)
attention-mechanism graph-attention-networks neural-networks python self-attention tensorflow
Language:Python 3452
Diego999 / pyGAT
Pytorch implementation of the Graph Attention Network model by Veličković et. al (2017, https://arxiv.org/abs/1710.10903)
attention-mechanism graph-attention-networks neural-networks python pytorch self-attention
Language:Python 3079
pytorch-GAT
gordicaleksa / pytorch-GAT
My implementation of the original GAT paper (Veličković et al.). I've additionally included the playground.py file for visualizing the Cora dataset, GAT embeddings, an attention mechanism, and entropy histograms. I've supported both Cora (transductive) and PPI (inductive) examples!
gat graph-attention-networks attention-mechanism self-attention pytorch python attention pytorch-gat gat-tutorial deep-learning graph-attention-network pytorch-implementation jupyter
Language:Jupyter Notebook 2617
github / CodeSearchNet
Datasets, tools, and benchmarks for representation learning of code.
bert cnn data data-science datasets deep-learning machine-learning machine-learning-on-source-code ml natural-language-processing neural-networks nlp nlp-machine-learning open-data programming-language-theory python representation-learning rnn self-attention tensorflow
Language:Jupyter Notebook 2385
microsoft / DeBERTa
The implementation of DeBERTa
bert deeplearning language-model natural-language-understanding representation-learning roberta self-attention transformer-encoder
Language:Python 2162
NVlabs / MambaVision
[CVPR 2025] Official PyTorch Implementation of MambaVision: A Hybrid Mamba-Transformer Vision Backbone
deep-learning foundation-models huggingface-transformers hybrid-models image-classification instance-segmentation mamba object-detection self-attention semantic-segmentation transformers vision-transformer visual-recognition
Language:Python 1843
speedinghzl / CCNet
CCNet: Criss-Cross Attention for Semantic Segmentation (TPAMI 2020 & ICCV 2019).
cityscape pytorch scene-parsing segmentation self-attention semantic-segmentation
Language:Python 1477
DirtyHarryLYL / Transformer-in-Vision
Recent Transformer-based CV and related works.
transformer vision-transformers computer-vision self-attention multi-modal visual-language deep-learning paper
1334
self-attention-cv
The-AI-Summer / self-attention-cv
Implementation of various self-attention mechanisms focused on computer vision. Ongoing repository.
artificial-intelligence attention attention-mechanism deep-learning machine-learning machine-learning-algorithms self-attention transformer transformers
Language:Python 1210
awesome-fast-attention
Separius / awesome-fast-attention
list of efficient attention modules
transformer attention awesome reformer longformer linformer multihead-attention self-attention attention-is-all-you-need transformer-network
Language:Python 1016
brightmart / bert_language_understanding
Pre-training of Deep Bidirectional Transformers for Language Understanding: pre-train TextCNN
bert-model language-model attention-is-all-you-need transformer-encoder pre-training language-understanding text-classification document-classification self-attention transfer-learning nlp question-answering textcnn fasttext
Language:Python 966
NVlabs / FasterViT
[ICLR 2024] Official PyTorch implementation of FasterViT: Fast Vision Transformers with Hierarchical Attention
ade20k backbone coco deep-learning foundation-models image-classification image-net object-detection pre-trained-model self-attention semantic-segmentation vision-transformer visual-recognition
Language:Python 886
xxxnell / how-do-vits-work
(ICLR 2022 Spotlight) Official PyTorch implementation of "How Do Vision Transformers Work?"
vision-transformer transformer self-attention loss-landscape pytorch
Language:Python 818
prakashpandey9 / Text-Classification-Pytorch
Text classification using deep learning models in Pytorch
pytorch text-classification sentiment-classification lstm-model attention-model self-attention rnn-model glove
Language:Python 816
kaituoxu / Speech-Transformer
A PyTorch implementation of Speech Transformer, an End-to-End ASR with Transformer network on Mandarin Chinese.
asr transformer end-to-end attention self-attention attention-is-all-you-need pytorch
Language:Python 803
daiquocnguyen / Graph-Transformer
Universal Graph Transformer Self-Attention Networks (TheWebConf WWW 2022) (Pytorch and Tensorflow)
graph-classification graph-deep-learning graph-embeddings graph-machine-learning graph-neural-networks graph-representation-learning graph-transformer node-embeddings self-attention text-classification transformer transformer-models
Language:Python 677
jayparks / transformer
A Pytorch Implementation of "Attention is All You Need" and "Weighted Transformer Network for Machine Translation"
self-attention attention-mechanism attention-is-all-you-need machine-translation
Language:Python 562
kaushalshetty / Structured-Self-Attention
A Structured Self-attentive Sentence Embedding
attention-mechanism attention-model self-attention self-attentive-rnn pytorch deep-learning python3 attention visualization classification sentence-embeddings attention-weights
Language:Python 494
NVlabs / FAN
Official PyTorch implementation of Fully Attentional Networks
backbone deep-learning image-classification object-detection vision-transformers visual-recognition semantic-segmentation corruption imagenet out-of-distribution pre-train cityscapes coco information-bottleneck self-attention visual-grouping
Language:Python 480
SAITS
WenjieDu / SAITS
The official PyTorch implementation of the paper "SAITS: Self-Attention-based Imputation for Time Series". A fast and state-of-the-art (SOTA) deep-learning neural network model for efficient time-series imputation (impute multivariate incomplete time series containing NaN missing data/values with machine learning). https://arxiv.org/abs/2202.08516
time-series imputation-model missing-values self-attention partially-observed-data partially-observed-time-series partially-observed interpolation time-series-imputation incomplete-data incomplete-time-series imputation impute pytorch transformer attention attention-mechanism irregular-sampling deep-learning machine-learning
Language:Python 470
leaderj1001 / Stand-Alone-Self-Attention
Implementing Stand-Alone Self-Attention in Vision Models using Pytorch
stand-alone-self-attention self-attention pytorch
Language:Python 455
binli123 / dsmil-wsi
DSMIL: Dual-stream multiple instance learning networks for tumor detection in Whole Slide Image
deep-learning multiple-instance-learning whole-slide-imaging whole-slide-image tumor-detection self-attention weakly-supervised-learning histopathology deep-neural-networks semi-supervised-learning self-supervised-learning pytorch transformer visiontransformer
Language:Python 444
Tixierae / deep_learning_NLP
Keras, PyTorch, and NumPy Implementations of Deep Learning Architectures for NLP
python3 keras deep-learning saliency-map cnn-keras nlp self-attention hierarchical-attention-network recurrent-neural-networks seq2seq-pytorch nmt skipgram word2vec pytorch attention numpy doc2vec word-embeddings document-embeddings
Language:Jupyter Notebook 442
NVlabs / GCVit
[ICML 2023] Official PyTorch implementation of Global Context Vision Transformers
vision-transformer ade20k coco deep-learning imagenet imagenet-classification object-detection pre-train pre-trained-model self-attention semantic-segmentation backbone visual-recognition
Language:Python 440
flash-dmattn
SmallDoges / flash-dmattn
Trainable fast and memory-efficient sparse attention
attention-is-all-you-need attention-mechanism chinese cuda-kernels cutlass dynamic-mask-attention english flash-attention pytorch pytorch-implementation self-attention transformer transformers triton triton-kernels
Language:C++ 418
naver-ai / rope-vit
[ECCV 2024] Official PyTorch implementation of RoPE-ViT "Rotary Position Embedding for Vision Transformer"
eccv2024 position-embedding rotary-position-embedding self-attention transformer vision-transformer
Language:Python 408
jw9730 / tokengt
[NeurIPS'22] Tokenized Graph Transformer (TokenGT), in PyTorch
equivariance gnn graph hypergraph pytorch self-attention transformer
Language:Python 341
fudan-zvg / SOFT
[NeurIPS 2021 Spotlight] & [IJCV 2024] SOFT: Softmax-free Transformer with Linear Complexity
linear-complexity linear-transformer self-attention softmax-free transformer
Language:Python 310
HyperSIGMA
WHU-Sigma / HyperSIGMA
The official repo for [TPAMI'25] "HyperSIGMA: Hyperspectral Intelligence Comprehension Foundation Model"
computer-vision deep-learning hyperspectral-anomaly-detection hyperspectral-datasets hyperspectral-image-classification hyperspectral-image-denoising hyperspectral-image-segmentation hyperspectral-unmixing pytorch remote-sensing self-attention spectral-spatial-information-fusion transformer hyperspectral-change-detection hyperspectral-super-resolution hyperspectral-target-detection foundation-models pre-training foundation-model remote-sensing-foundation-model
Language:Python 306
aravindsankar28 / DySAT
Representation learning on dynamic graphs using self-attention networks
dynamic-graphs self-attention graph-neural-network graph-embedding
Language:Python 302
wangxiao5791509 / MultiModal_BigModels_Survey
[MIR-2023-Survey] A continuously updated paper list for multi-modal pre-trained big models
audio depth event-camera multi-modal natural-language pengchenglab point-cloud pre-training radar review self-attention survey thermal-infrared transformers anhui-university big-models rgb-text-audio
288
wenwenyu / MASTER-pytorch
Code for the paper "MASTER: Multi-Aspect Non-local Network for Scene Text Recognition" (Pattern Recognition 2021)
scene-text-recognition transformer self-attention non-local-network ocr
Language:Python 280
alohays / awesome-visual-representation-learning-with-transformers
Awesome Transformers (self-attention) in Computer Vision
transformers self-attention computer-vision awesome-list representation-learning vision-transformer
270

self-attention

datawhalechina / leedl-tutorial

zhouhaoyi / Informer2020

cmhungsteve / Awesome-Transformer-Attention

PetarV- / GAT

Diego999 / pyGAT

gordicaleksa / pytorch-GAT

github / CodeSearchNet

microsoft / DeBERTa

NVlabs / MambaVision

speedinghzl / CCNet

DirtyHarryLYL / Transformer-in-Vision

The-AI-Summer / self-attention-cv

Separius / awesome-fast-attention

brightmart / bert_language_understanding

NVlabs / FasterViT

xxxnell / how-do-vits-work

prakashpandey9 / Text-Classification-Pytorch

kaituoxu / Speech-Transformer

daiquocnguyen / Graph-Transformer

jayparks / transformer

kaushalshetty / Structured-Self-Attention

NVlabs / FAN

WenjieDu / SAITS

leaderj1001 / Stand-Alone-Self-Attention

binli123 / dsmil-wsi

Tixierae / deep_learning_NLP

NVlabs / GCVit

SmallDoges / flash-dmattn

naver-ai / rope-vit

jw9730 / tokengt

fudan-zvg / SOFT

WHU-Sigma / HyperSIGMA

aravindsankar28 / DySAT

wangxiao5791509 / MultiModal_BigModels_Survey

wenwenyu / MASTER-pytorch

alohays / awesome-visual-representation-learning-with-transformers