multi-head-attention

There are 3 repositories under multi-head-attention topic.

anicolson / DeepXi
Deep Xi: A deep learning approach to a priori SNR estimation implemented in TensorFlow 2/Keras. For speech enhancement and robust ASR.
resnet tensorflow speech-enhancement robust-asr deepxi a-priori-snr-estimator mmse minimum-mean-square-error mmse-lsa residual-networks deep-xi speech-separation noise-estimation source-separation deepmmse tcn keras attention mhanet multi-head-attention
Language:MATLAB 486
sooftware / attentions
PyTorch implementation of some attentions for Deep Learning Researchers.
additive-attention attention dot-product-attention location-aware-attention location-sensitive-attension multi-head-attention pytorch relative-multi-head-attention relative-positional-encoding
Language:Python 481
imperial-qore / TranAD
[VLDB'22] Anomaly Detection using Transformers, self-conditioning and adversarial training.
adversarial-learning anomaly-detection multi-head-attention transformer-models unsupervised-learning
Language:Python 464
dodrio
poloclub / dodrio
Exploring attention weights in transformer-based models with linguistic knowledge.
visualization deep-learning transformer attention-mechanism multi-head-attention interactive-visualizations nlp
Language:Svelte 326
Rintarooo / VRP_DRL_MHA
"Attention, Learn to Solve Routing Problems!"[Kool+, 2019], Capacitated Vehicle Routing Problem solver
multi-head-attention vrp deep-reinforcement-learning pytorch tensorflow capacitated-vehicle-routing-problem policy-gradient reinforce
Language:Python 153
monk1337 / Various-Attention-mechanisms
This repository contain various types of attention mechanism like Bahdanau , Soft attention , Additive Attention , Hierarchical Attention etc in Pytorch, Tensorflow, Keras
attention-mechanism attention keras pytorch attention-model attention-mechanisms attention-lstm attention-network bahdanau-attention hierarchical-attention luong-attention multi-head-attention scaled-dot-product-attention self-attention sentence-attention
Language:Python 118
datnnt1997 / multi-head_self-attention
A Faster Pytorch Implementation of Multi-Head Self-Attention
attention attention-mechanism multi-head multi-head-attention multi-head-self-attention multihead-attention multihead-self-attention pytorch-self-attention self-attention transformer-attention
Language:Jupyter Notebook 67
zhaocq-nlp / Attention-Visualization
Visualization for simple attention and Google's multi-head attention.
attention multi-head-attention neural-machine-translation machine-translation visualization attention-visualization
Language:Java 65
youngbin-ro / Multi2OIE
Multi^2OIE: Multilingual Open Information Extraction Based on Multi-Head Attention with BERT (Findings of ACL: EMNLP 2020)
information-extraction multi-head-attention bert open-information-extraction multilingualism sequence-labeling
Language:Python 57
ShaneTian / Att-Induction
Attention-based Induction Networks for Few-Shot Text Classification
few-shot-learning text-classification bert multi-head-attention neural-tensor-network
Language:Python 44
engelnico / point-transformer
This is the official repository of the original Point Transformer architecture.
3d-point-cloud 3d-pointclouds 3d-classification pytorch deep-learning transformer-architecture 3d-segmentation classification segmentation modelnet-dataset modelnet modelnet40 shapenet-dataset shapenetpart transformer attention-mechanism multi-head-attention sortnet
Language:Python 35
JacobHanimann / scDINO
Self-Supervised Vision Transformers for multiplexed imaging datasets
high-content-screening self-supervised-learning single-cell fluorescence-microscopy-imaging greyscale-image morphology multi-channel multi-head-attention phenotyping vision-transformer unsupervised
Language:Python 32
knotgrass / attention
several types of attention modules written in PyTorch
attention attention-mechanism multi-head-attention multi-query-attention pytorch transformer transformers grouped-query-attention scale-dot-product-attention softmax-layer
Language:Python 27
shifop / datagrand_bert
2019达观杯信息提取第5名代码
bert multi-head-attention ner
Language:Python 20
jack57lee / Diversify-MHA
EMNLP 2018: Multi-Head Attention with Disagreement Regularization; NAACL 2019: Information Aggregation for Multi-Head Attention with Routing-by-Agreement
neural-machine-transliteration deep-learning multi-head-attention
Language:Python 18
Zminghua / SentEncoding
Sentence encoder and training code for Mean-Max AAE
multi-head-attention representation
Language:Python 16
M-e-r-c-u-r-y / pytorch-transformers
Collection of different types of transformers for learning purposes
pytorch transformers multi-head-attention multi-query-attention einsum-notation
Language:Jupyter Notebook 13
shreyas-kowshik / nlp4if
Code for the runners up entry on the English subtask on the Shared-Task-On-Fighting the COVID-19 Infodemic, NLP4IF workshop, NAACL'21.
natural-language-processing deep-learning multi-head-attention naacl2021 multi-task-learning
Language:Python 6
pi-tau / transformer
The Transformer model implemented from scratch using PyTorch. The model uses weight sharing between the embedding layers and the pre-softmax linear layer. Training on the Multi30k machine translation task is shown.
deep-learning multi-head-attention pytorch transformer machine-translation multi30k shared-embedding
Language:Python 4
tanishqgautam / Transformers
Pytorch Implementation of Transformers
deep-learning multi-head-attention nlp pytorch self-attention transformer
Language:Python 4
TranQuocTrinh / image_captioning
Image Captioning with Encoder as Efficientnet and Decoder as Decoder of Transformer combined with the attention mechanism.
python pytorch deep-learning image-captioning transformer efficientnet multi-head-attention attention-mechanism natural-language-processing convolutional-neural-networks
Language:Python 4
YigitTurali / HydraViT
HydraViT is a PyTorch implementation of the HydraViT model, an adaptive multi-branch transformer for multi-label disease classification from chest X-ray images. The repository provides the necessary code to train and evaluate the HydraViT model on the NIH Chest X-ray dataset.
chest-xray-images computer-vision deep-learning machine-learning multi-head-attention neural-networks visual-transformers
4
AIMedLab / DeepCE
Code and Datasets for the paper "A deep learning framework for high-throughput mechanism-driven phenotype compound screening and its application to COVID-19 drug repurposing", published on Nature Machine Intelligence in 2021.
phenotypic-drug-discovery graph-neural-network multi-head-attention l1000 covid-19 drug-repurposing chemical-induced-gene-expression
Language:Python 3
gazelle93 / Attention-Various-Positional-Encoding
This project aims to implement the Scaled-Dot-Product Attention layer and the Multi-Head Attention layer using various Positional Encoding methods.
attention-mechanism gensim multi-head-attention natural-language-processing nlp nltk pytorch relative-positional-encoding relative-positional-representation scaled-dot-product spacy t5 wordembeddings
Language:Python 2
liaoyanqing666 / transformer_pytorch
完整的原版transformer程序，complete origin transformer program
beginner multi-head-attention positional-encoding python pytorch transformer
Language:Python 1
navreeetkaur / learn-to-pay-attention
TensorFlow implementation of AlexNet with multi-headed Attention mechanism
tensorflow alexnet attention-mechanism attention-model multi-head-attention
Language:Jupyter Notebook 1
SpydazWebAI-NLP / BasicNeuralNetWork2023
A Basic Multi layered Neural Network, With Attention Masking Features
multi-head-attention neural-network nlp rnn self-attention transformer-architecture
Language:Visual Basic .NET 1
young-zonglin / yangzl-deep-text-matching
Text matching using several deep models.
text-matching deep-model transformer-encoder rnmt-plus-encoder deep-lstms multi-head-attention
Language:Python 1
Transformer_NMT
AshishBodhankar / Transformer_NMT
Attention is all you need: Discovering the Transformer model
transformer attention vaswani machine-translation natural-language-processing multi-head-attention mask
Language:Jupyter Notebook 0
tate8 / translator
Transformer translator website with multithreaded web server in Rust
chatbot css html javascript keras machine-learning multi-head-attention multi-threading positional-encoding python rust tcp tensorflow thread-pool transformer web-server website word-embedding
Language:Rust 0
TmohamedashrafT / vision-transformer-implementation
This repository contains code for implementing Vision Transformer (ViT) model for image classification
multi-head-attention transformers vision-transformer
Language:Python 0
sajith-rahim / transformer-classifier
A Transformer Classifier implemented from Scratch.
classification-model attention multi-head-attention sentence-classification embeddings pytorch scratch-implementation
Language:Python

multi-head-attention

anicolson / DeepXi

sooftware / attentions

imperial-qore / TranAD

poloclub / dodrio

Rintarooo / VRP_DRL_MHA

monk1337 / Various-Attention-mechanisms

datnnt1997 / multi-head_self-attention

zhaocq-nlp / Attention-Visualization

youngbin-ro / Multi2OIE

ShaneTian / Att-Induction

engelnico / point-transformer

JacobHanimann / scDINO

knotgrass / attention

shifop / datagrand_bert

jack57lee / Diversify-MHA

Zminghua / SentEncoding

M-e-r-c-u-r-y / pytorch-transformers

shreyas-kowshik / nlp4if

pi-tau / transformer

tanishqgautam / Transformers

TranQuocTrinh / image_captioning

YigitTurali / HydraViT

AIMedLab / DeepCE

gazelle93 / Attention-Various-Positional-Encoding

liaoyanqing666 / transformer_pytorch

navreeetkaur / learn-to-pay-attention

SpydazWebAI-NLP / BasicNeuralNetWork2023

young-zonglin / yangzl-deep-text-matching

AshishBodhankar / Transformer_NMT

tate8 / translator

TmohamedashrafT / vision-transformer-implementation

sajith-rahim / transformer-classifier