self-attention

There are 21 repositories under self-attention topic.

datawhalechina / leedl-tutorial
《李宏毅深度学习教程》（李宏毅老师推荐👍），冲击万星，感谢大家Star :star:！PDF下载地址：https://github.com/datawhalechina/leedl-tutorial/releases
machine-learning deep-learning leedl-tutorial cnn reinforcement-learning transformer rnn gan pruning self-attention chatgpt tutorial network-compression diffusion transfer-learning bert
Language:Jupyter Notebook 9915
zhouhaoyi / Informer2020
The GitHub repository for the paper "Informer" accepted by AAAI 2021.
deep-learning self-attention time-series pytorch forecasting transformer
Language:Python 4964
cmhungsteve / Awesome-Transformer-Attention
An ultimately comprehensive paper list of Vision Transformer/Attention, including papers, codes, and related websites
transformer attention-mechanism vision-transformer deep-learning awesome-list transformer-cv transformer-architecture transformer-awesome transformer-with-cv transformer-models visual-transformer computer-vision papers attention-mechanisms self-attention vit detr transformers
4292
PetarV- / GAT
Graph Attention Networks (https://arxiv.org/abs/1710.10903)
graph-attention-networks attention-mechanism self-attention tensorflow neural-networks python
Language:Python 3081
Diego999 / pyGAT
Pytorch implementation of the Graph Attention Network model by Veličković et. al (2017, https://arxiv.org/abs/1710.10903)
graph-attention-networks attention-mechanism self-attention pytorch neural-networks python
Language:Python 2797
pytorch-GAT
gordicaleksa / pytorch-GAT
My implementation of the original GAT paper (Veličković et al.). I've additionally included the playground.py file for visualizing the Cora dataset, GAT embeddings, an attention mechanism, and entropy histograms. I've supported both Cora (transductive) and PPI (inductive) examples!
gat graph-attention-networks attention-mechanism self-attention pytorch python attention pytorch-gat gat-tutorial deep-learning graph-attention-network pytorch-implementation jupyter
Language:Jupyter Notebook 2312
github / CodeSearchNet
Datasets, tools, and benchmarks for representation learning of code.
deep-learning natural-language-processing programming-language-theory machine-learning tensorflow datasets data-science machine-learning-on-source-code representation-learning neural-networks python ml open-data data cnn rnn self-attention bert nlp nlp-machine-learning
Language:Jupyter Notebook 2124
microsoft / DeBERTa
The implementation of DeBERTa
bert deeplearning representation-learning roberta language-model natural-language-understanding self-attention transformer-encoder
Language:Python 1865
speedinghzl / CCNet
CCNet: Criss-Cross Attention for Semantic Segmentation (TPAMI 2020 & ICCV 2019).
cityscape semantic-segmentation self-attention pytorch scene-parsing segmentation
Language:Python 1397
DirtyHarryLYL / Transformer-in-Vision
Recent Transformer-based CV and related works.
transformer vision-transformers computer-vision self-attention multi-modal visual-language deep-learning paper
1294
self-attention-cv
The-AI-Summer / self-attention-cv
Implementation of various self-attention mechanisms focused on computer vision. Ongoing repository.
deep-learning transformer transformers self-attention attention-mechanism attention machine-learning machine-learning-algorithms artificial-intelligence
Language:Python 1141
awesome-fast-attention
Separius / awesome-fast-attention
list of efficient attention modules
transformer attention awesome reformer longformer linformer multihead-attention self-attention attention-is-all-you-need transformer-network
Language:Python 978
brightmart / bert_language_understanding
Pre-training of Deep Bidirectional Transformers for Language Understanding: pre-train TextCNN
bert-model language-model attention-is-all-you-need transformer-encoder pre-training language-understanding text-classification document-classification self-attention transfer-learning nlp question-answering textcnn fasttext
Language:Python 959
prakashpandey9 / Text-Classification-Pytorch
Text classification using deep learning models in Pytorch
pytorch text-classification sentiment-classification lstm-model attention-model self-attention rnn-model glove
Language:Python 802
xxxnell / how-do-vits-work
(ICLR 2022 Spotlight) Official PyTorch implementation of "How Do Vision Transformers Work?"
vision-transformer transformer self-attention loss-landscape pytorch
Language:Python 790
kaituoxu / Speech-Transformer
A PyTorch implementation of Speech Transformer, an End-to-End ASR with Transformer network on Mandarin Chinese.
asr transformer end-to-end attention self-attention attention-is-all-you-need pytorch
Language:Python 764
NVlabs / FasterViT
[ICLR 2024] Official PyTorch implementation of FasterViT: Fast Vision Transformers with Hierarchical Attention
ade20k backbone deep-learning image-net pre-trained-model self-attention vision-transformer visual-recognition coco object-detection semantic-segmentation foundation-models image-classification
Language:Python 693
daiquocnguyen / Graph-Transformer
Universal Graph Transformer Self-Attention Networks (TheWebConf WWW 2022) (Pytorch and Tensorflow)
graph-representation-learning graph-classification graph-neural-networks node-embeddings graph-embeddings graph-transformer self-attention transformer graph-machine-learning text-classification transformer-models graph-deep-learning
Language:Python 623
jayparks / transformer
A Pytorch Implementation of "Attention is All You Need" and "Weighted Transformer Network for Machine Translation"
attention-is-all-you-need attention-mechanism machine-translation self-attention
Language:Python 534
kaushalshetty / Structured-Self-Attention
A Structured Self-attentive Sentence Embedding
attention-mechanism attention-model self-attention self-attentive-rnn pytorch deep-learning python3 attention visualization classification sentence-embeddings attention-weights
Language:Python 493
NVlabs / FAN
Official PyTorch implementation of Fully Attentional Networks
backbone deep-learning image-classification object-detection vision-transformers visual-recognition semantic-segmentation corruption imagenet out-of-distribution pre-train cityscapes coco information-bottleneck self-attention visual-grouping
Language:Python 461
leaderj1001 / Stand-Alone-Self-Attention
Implementing Stand-Alone Self-Attention in Vision Models using Pytorch
stand-alone-self-attention self-attention pytorch
Language:Python 455
Tixierae / deep_learning_NLP
Keras, PyTorch, and NumPy Implementations of Deep Learning Architectures for NLP
python3 keras deep-learning saliency-map cnn-keras nlp self-attention hierarchical-attention-network recurrent-neural-networks seq2seq-pytorch nmt skipgram word2vec pytorch attention numpy doc2vec word-embeddings document-embeddings
Language:Jupyter Notebook 435
NVlabs / GCVit
[ICML 2023] Official PyTorch implementation of Global Context Vision Transformers
vision-transformer ade20k coco deep-learning imagenet imagenet-classification object-detection pre-train pre-trained-model self-attention semantic-segmentation backbone visual-recognition
Language:Python 415
binli123 / dsmil-wsi
DSMIL: Dual-stream multiple instance learning networks for tumor detection in Whole Slide Image
deep-learning multiple-instance-learning whole-slide-imaging whole-slide-image tumor-detection self-attention weakly-supervised-learning histopathology deep-neural-networks semi-supervised-learning self-supervised-learning pytorch transformer visiontransformer
Language:Python 318
jw9730 / tokengt
[NeurIPS'22] Tokenized Graph Transformer (TokenGT), in PyTorch
equivariance gnn graph hypergraph pytorch self-attention transformer
Language:Python 303
fudan-zvg / SOFT
[NeurIPS 2021 Spotlight] & [IJCV 2024] SOFT: Softmax-free Transformer with Linear Complexity
transformer linear-transformer self-attention linear-complexity softmax-free
Language:Python 294
wenwenyu / MASTER-pytorch
Code for the paper "MASTER: Multi-Aspect Non-local Network for Scene Text Recognition" (Pattern Recognition 2021)
non-local-network ocr scene-text-recognition self-attention transformer
Language:Python 276
aravindsankar28 / DySAT
Representation learning on dynamic graphs using self-attention networks
dynamic-graphs self-attention graph-neural-network graph-embedding
Language:Python 272
kushalj001 / pytorch-question-answering
Important paper implementations for Question Answering using PyTorch
nlp-machine-learning question-answering pytorch-implementation pytorch-tutorial squad-dataset deep-learning attention-mechanism self-attention convolutional-neural-networks nlp natural-language-processing tutorial bidaf-pytorch bidaf drqa qanet pytorch paper-implementations
Language:Jupyter Notebook 272
SAITS
WenjieDu / SAITS
The official PyTorch implementation of the paper "SAITS: Self-Attention-based Imputation for Time Series". A fast and state-of-the-art (SOTA) deep-learning neural network model for efficient time-series imputation (impute multivariate incomplete time series containing NaN missing data/values with machine learning). https://arxiv.org/abs/2202.08516
time-series imputation-model missing-values self-attention partially-observed-data partially-observed-time-series partially-observed interpolation time-series-imputation incomplete-data incomplete-time-series imputation impute pytorch transformer attention attention-mechanism irregular-sampling deep-learning machine-learning
Language:Python 269
alohays / awesome-visual-representation-learning-with-transformers
Awesome Transformers (self-attention) in Computer Vision
transformers self-attention computer-vision awesome-list representation-learning vision-transformer
263
wangxiao5791509 / MultiModal_BigModels_Survey
[MIR-2023-Survey] A continuously updated paper list for multi-modal pre-trained big models
anhui-university audio big-models depth event-camera multi-modal natural-language pengchenglab point-cloud pre-training radar review rgb-text-audio self-attention survey thermal-infrared transformers
252
cbaziotis / neat-vision
Neat (Neural Attention) Vision, is a visualization tool for the attention mechanisms of deep-learning models for Natural Language Processing (NLP) tasks. (framework-agnostic)
attention attention-mechanism attention-visualization attention-scores text-visualization deep-learning deep-learning-visualization vuejs nlp natural-language-processing attention-mechanisms visualization deep-learning-library self-attention self-attentive-rnn
Language:Vue 249
L0SG / relational-rnn-pytorch
An implementation of DeepMind's Relational Recurrent Neural Networks (NeurIPS 2018) in PyTorch.
pytorch language-model word-language-model language-modeling deep-learning recurrent-neural-networks deepmind transformer self-attention
Language:Python 246
flrngel / Self-Attentive-tensorflow
Tensorflow implementation of "A Structured Self-Attentive Sentence Embedding"
self-attention tensorflow attention self-attentive-rnn
Language:Python 192

self-attention

datawhalechina / leedl-tutorial

zhouhaoyi / Informer2020

cmhungsteve / Awesome-Transformer-Attention

PetarV- / GAT

Diego999 / pyGAT

gordicaleksa / pytorch-GAT

github / CodeSearchNet

microsoft / DeBERTa

speedinghzl / CCNet

DirtyHarryLYL / Transformer-in-Vision

The-AI-Summer / self-attention-cv

Separius / awesome-fast-attention

brightmart / bert_language_understanding

prakashpandey9 / Text-Classification-Pytorch

xxxnell / how-do-vits-work

kaituoxu / Speech-Transformer

NVlabs / FasterViT

daiquocnguyen / Graph-Transformer

jayparks / transformer

kaushalshetty / Structured-Self-Attention

NVlabs / FAN

leaderj1001 / Stand-Alone-Self-Attention

Tixierae / deep_learning_NLP

NVlabs / GCVit

binli123 / dsmil-wsi

jw9730 / tokengt

fudan-zvg / SOFT

wenwenyu / MASTER-pytorch

aravindsankar28 / DySAT

kushalj001 / pytorch-question-answering

WenjieDu / SAITS

alohays / awesome-visual-representation-learning-with-transformers

wangxiao5791509 / MultiModal_BigModels_Survey

cbaziotis / neat-vision

L0SG / relational-rnn-pytorch

flrngel / Self-Attentive-tensorflow