attention-mechanism

There are 42 repositories under attention-mechanism topic.

lucidrains / vit-pytorch
Implementation of Vision Transformer, a simple way to achieve SOTA in vision classification with only a single transformer encoder, in Pytorch
artificial-intelligence attention-mechanism transformers computer-vision image-classification
Language:Python 18198
BlinkDL / RWKV-LM
RWKV is an RNN with transformer-level LLM performance. It can be directly trained like a GPT (parallelizable). So it's combining the best of RNN and transformer - great performance, fast inference, saves VRAM, fast training, "infinite" ctx_len, and free sentence embedding.
attention-mechanism chatgpt deep-learning gpt gpt-2 gpt-3 language-model linear-attention lstm pytorch rnn rwkv transformer transformers
Language:Python 11734
brightmart / text_classification
all kinds of text classification models and more with deep learning
classification nlp fasttext textcnn textrnn tensorflow multi-label multi-class attention-mechanism text-classification convolutional-neural-networks sentence-classification memory-networks
Language:Python 7749
lucidrains / DALLE-pytorch
Implementation / replication of DALL-E, OpenAI's Text to Image Transformer, in Pytorch
artificial-intelligence attention-mechanism deep-learning multi-modal text-to-image transformers
Language:Python 5493
awesome-graph-classification
benedekrozemberczki / awesome-graph-classification
A collection of important graph embedding, classification and representation learning papers with implementations.
graph2vec classification-algorithm graph-kernels weisfeiler-lehman kernel-methods deep-graph-kernels netlsd graph-attention-model graph-attention-networks structural-attention attention-mechanism graph-kernel node2vec graph-embedding network-embedding node-embedding graph-representation-learning deepwalk graph-convolutional-networks graph-classification
Language:Python 4705
cmhungsteve / Awesome-Transformer-Attention
An ultimately comprehensive paper list of Vision Transformer/Attention, including papers, codes, and related websites
transformer attention-mechanism vision-transformer deep-learning awesome-list transformer-cv transformer-architecture transformer-awesome transformer-with-cv transformer-models visual-transformer computer-vision papers attention-mechanisms self-attention vit detr transformers
4279
lucidrains / x-transformers
A simple but complete full-attention transformer with a set of promising experimental features from various papers
artificial-intelligence attention-mechanism deep-learning transformers
Language:Python 4192
Kyubyong / transformer
A TensorFlow Implementation of the Transformer: Attention Is All You Need
attention-mechanism translation attention-is-all-you-need implementation transformer
Language:Python 4173
PetarV- / GAT
Graph Attention Networks (https://arxiv.org/abs/1710.10903)
attention-mechanism graph-attention-networks neural-networks python self-attention tensorflow
Language:Python 3079
zzw922cn / awesome-speech-recognition-speech-synthesis-papers
Automatic Speech Recognition (ASR), Speaker Verification, Speech Synthesis, Text-to-Speech (TTS), Language Modelling, Singing Voice Synthesis (SVS), Voice Conversion (VC)
automatic-speech-recognition papers roadmap rnn cnn dnn attention-mechanism seq2seq acoustic-model timit-dataset tts language-model speaker-verification speech-recognition speech-synthesis neural-network recognition-synthesis diffusion-models singing-voice-synthesis voice-conversion
2879
philipperemy / keras-attention
Keras Attention Layer (Luong and Bahdanau scores).
attention-mechanism attention-model deep-learning keras keras-neural-networks
Language:Python 2800
Diego999 / pyGAT
Pytorch implementation of the Graph Attention Network model by Veličković et. al (2017, https://arxiv.org/abs/1710.10903)
attention-mechanism graph-attention-networks neural-networks python pytorch self-attention
Language:Python 2794
sgrvinod / a-PyTorch-Tutorial-to-Image-Captioning
Show, Attend, and Tell | a PyTorch Tutorial to Image Captioning
pytorch pytorch-tutorial show-attend-and-tell image-captioning encoder-decoder attention-mechanism computer-vision mscoco
Language:Python 2665
pytorch-GAT
gordicaleksa / pytorch-GAT
My implementation of the original GAT paper (Veličković et al.). I've additionally included the playground.py file for visualizing the Cora dataset, GAT embeddings, an attention mechanism, and entropy histograms. I've supported both Cora (transductive) and PPI (inductive) examples!
gat graph-attention-networks attention-mechanism self-attention pytorch python attention pytorch-gat gat-tutorial deep-learning graph-attention-network pytorch-implementation jupyter
Language:Jupyter Notebook 2308
lucidrains / reformer-pytorch
Reformer, the efficient Transformer, in Pytorch
artificial-intelligence transformers attention-mechanism machine-learning pytorch
Language:Python 2058
linto-ai / whisper-timestamped
Multilingual Automatic Speech Recognition with word-level timestamps and confidence
deep-learning speech speech-recognition speech-to-text asr machine-learning python python3 pytorch attention-is-all-you-need attention-mechanism attention-model attention-network attention-seq2seq attention-visualization multilingual-models speaker-diarization speech-processing transformers whisper
Language:Python 1559
lucidrains / lambda-networks
Implementation of LambdaNetworks, a new approach to image recognition that reaches SOTA with less compute
artificial-intelligence deep-learning computer-vision attention-mechanism attention
Language:Python 1534
lucidrains / alphafold2
To eventually become an unofficial Pytorch implementation / replication of Alphafold2, as details of the architecture get released
artificial-intelligence attention-mechanism deep-learning protein-folding
Language:Python 1518
charlesXu86 / Chatbot_CN
基于金融-司法领域(兼有闲聊性质)的聊天机器人，其中的主要模块有信息抽取、NLU、NLG、知识图谱等，并且利用Django整合了前端展示,目前已经封装了nlp和kg的restful接口
deep-learning chatbot-cn tenserflow-serving intent-detection django-restful reinforcement-learning knowledge-graph slot-filling dialogue-systems ir oriented-dialogs sentiment-analysis tensorflow ner nlu nlg attention-mechanism text-correct text-classification recommendation
1270
awslabs / sockeye
Sequence-to-sequence framework with a focus on Neural Machine Translation based on PyTorch
deep-learning deep-neural-networks machine-learning machine-translation neural-machine-translation encoder-decoder attention-mechanism sequence-to-sequence sequence-to-sequence-models sockeye attention-is-all-you-need attention-model seq2seq translation transformer-architecture transformer transformer-network pytorch
Language:Python 1207
lucidrains / flamingo-pytorch
Implementation of 🦩 Flamingo, state-of-the-art few-shot visual question answering attention net out of Deepmind, in Pytorch
artificial-intelligence attention-mechanism deep-learning transformers visual-question-answering
Language:Python 1152
self-attention-cv
The-AI-Summer / self-attention-cv
Implementation of various self-attention mechanisms focused on computer vision. Ongoing repository.
artificial-intelligence attention attention-mechanism deep-learning machine-learning machine-learning-algorithms self-attention transformer transformers
Language:Python 1143
lucidrains / soundstorm-pytorch
Implementation of SoundStorm, Efficient Parallel Audio Generation from Google Deepmind, in Pytorch
artificial-intelligence attention-mechanism audio-generation deep-learning non-autoregressive transformers
Language:Python 1122
richliao / textClassifier
Text classifier for Hierarchical Attention Networks for Document Classification
attention-mechanism convolutional-neural-networks hierarchical-attention-networks recurrent-neural-networks text-classification
Language:Python 1062
lucidrains / performer-pytorch
An implementation of Performer, a linear attention-based transformer, in Pytorch
artificial-intelligence deep-learning attention-mechanism attention transformers
Language:Python 1058
lucidrains / perceiver-pytorch
Implementation of Perceiver, General Perception with Iterative Attention, in Pytorch
artificial-intelligence attention-mechanism deep-learning
Language:Python 1052
cedrickchee / awesome-transformer-nlp
A curated list of NLP resources focused on Transformer networks, attention mechanism, GPT, BERT, ChatGPT, LLMs, and transfer learning.
nlp natural-language-processing bert transformer language-model transfer-learning attention-mechanism awesome-list gpt-2 xlnet neural-networks pre-trained-language-models gpt-3 awesome chatgpt gpt-4 llama
1036
lucidrains / CoCa-pytorch
Implementation of CoCa, Contrastive Captioners are Image-Text Foundation Models, in Pytorch
artificial-intelligence attention-mechanism contrastive-learning deep-learning image-to-text multimodal transformers
Language:Python 979
pytorch-original-transformer
gordicaleksa / pytorch-original-transformer
My implementation of the original transformer model (Vaswani et al.). I've additionally included the playground.py file for visualizing otherwise seemingly hard concepts. Currently included IWSLT pretrained models.
transformer transformers pytorch-transformer pytorch-transformers attention attention-mechanism attention-is-all-you-need pytorch python jupyter transformer-tutorial deeplearning deep-learning original-transformer
Language:Jupyter Notebook 937
yunjey / show-attend-and-tell
TensorFlow Implementation of "Show, Attend and Tell"
tensorflow image-captioning show-attend-and-tell attention-mechanism mscoco-image-dataset
Language:Jupyter Notebook 905
atulkum / pointer_summarizer
pytorch implementation of "Get To The Point: Summarization with Pointer-Generator Networks"
nlp deep-learning attention-mechanism summarization pointer-networks pytorch-implmention seq2seq-attn
Language:Python 896
SuperBruceJia / EEG-DL
A Deep Learning library for EEG Tasks (Signals) Classification, based on TensorFlow.
deep-learning eeg-classification eeg-signals-processing tensorflow motor-imagery-classification eeg-data cnn rnn gcn one-shot-learning residual-learning densenet resnet graph-convolutional-neural-networks lstm gru attention-mechanism fully-convolutional-networks transformer transformers
Language:Python 866
lucidrains / RETRO-pytorch
Implementation of RETRO, Deepmind's Retrieval based Attention net, in Pytorch
artificial-intelligence deep-learning transformers attention-mechanism retrieval
Language:Python 832
lucidrains / PaLM-pytorch
Implementation of the specific Transformer architecture from PaLM - Scaling Language Modeling with Pathways
deep-learning transformers attention-mechanism artificial-general-intelligence
Language:Python 811
datalogue / keras-attention
Visualizing RNNs using the attention mechanism
recurrent-neural-networks attention-mechanism deep-learning deep-neural-networks machine-learning natural-language-processing translation
Language:Python 745
positive666 / yolo_research
based on yolo-high-level project (detect\pose\classify\segment\):include yolov5\yolov7\yolov8\ core ,improvement research ,SwintransformV2 and Attention Series. training skills, business customization, engineering deployment C
yolov5 deepstream pytorch object-detection attention-mechanism swintransformer custom-networ yolov7 yolov8
Language:Python 743

attention-mechanism

lucidrains / vit-pytorch

BlinkDL / RWKV-LM

brightmart / text_classification

lucidrains / DALLE-pytorch

benedekrozemberczki / awesome-graph-classification

cmhungsteve / Awesome-Transformer-Attention

lucidrains / x-transformers

Kyubyong / transformer

PetarV- / GAT

zzw922cn / awesome-speech-recognition-speech-synthesis-papers

philipperemy / keras-attention

Diego999 / pyGAT

sgrvinod / a-PyTorch-Tutorial-to-Image-Captioning

gordicaleksa / pytorch-GAT

lucidrains / reformer-pytorch

linto-ai / whisper-timestamped

lucidrains / lambda-networks

lucidrains / alphafold2

charlesXu86 / Chatbot_CN

awslabs / sockeye

lucidrains / flamingo-pytorch

The-AI-Summer / self-attention-cv

lucidrains / soundstorm-pytorch

richliao / textClassifier

lucidrains / performer-pytorch

lucidrains / perceiver-pytorch

cedrickchee / awesome-transformer-nlp

lucidrains / CoCa-pytorch

gordicaleksa / pytorch-original-transformer

yunjey / show-attend-and-tell

atulkum / pointer_summarizer

SuperBruceJia / EEG-DL

lucidrains / RETRO-pytorch

lucidrains / PaLM-pytorch

datalogue / keras-attention

positive666 / yolo_research