multi-head-attention

There are 3 repositories under multi-head-attention topic.

anicolson / DeepXi
Deep Xi: A deep learning approach to a priori SNR estimation implemented in TensorFlow 2/Keras. For speech enhancement and robust ASR.
resnet tensorflow speech-enhancement robust-asr deepxi a-priori-snr-estimator mmse minimum-mean-square-error mmse-lsa residual-networks deep-xi speech-separation noise-estimation source-separation deepmmse tcn keras attention mhanet multi-head-attention
Language:MATLAB 488
sooftware / attentions
PyTorch implementation of some attentions for Deep Learning Researchers.
pytorch attention location-sensitive-attension location-aware-attention additive-attention dot-product-attention multi-head-attention relative-positional-encoding relative-multi-head-attention
Language:Python 488
imperial-qore / TranAD
[VLDB'22] Anomaly Detection using Transformers, self-conditioning and adversarial training.
anomaly-detection transformer-models multi-head-attention unsupervised-learning adversarial-learning
Language:Python 468
dodrio
poloclub / dodrio
Exploring attention weights in transformer-based models with linguistic knowledge.
attention-mechanism deep-learning interactive-visualizations multi-head-attention nlp transformer visualization
Language:Svelte 329
Rintarooo / VRP_DRL_MHA
"Attention, Learn to Solve Routing Problems!"[Kool+, 2019], Capacitated Vehicle Routing Problem solver
multi-head-attention vrp deep-reinforcement-learning pytorch tensorflow capacitated-vehicle-routing-problem policy-gradient reinforce
Language:Python 161
monk1337 / Various-Attention-mechanisms
This repository contain various types of attention mechanism like Bahdanau , Soft attention , Additive Attention , Hierarchical Attention etc in Pytorch, Tensorflow, Keras
attention-mechanism attention keras pytorch attention-model attention-mechanisms attention-lstm attention-network bahdanau-attention hierarchical-attention luong-attention multi-head-attention scaled-dot-product-attention self-attention sentence-attention
Language:Python 118
datnnt1997 / multi-head_self-attention
A Faster Pytorch Implementation of Multi-Head Self-Attention
self-attention attention-mechanism attention multihead-attention multihead-self-attention multi-head-self-attention pytorch-self-attention transformer-attention multi-head multi-head-attention
Language:Jupyter Notebook 68
zhaocq-nlp / Attention-Visualization
Visualization for simple attention and Google's multi-head attention.
attention multi-head-attention neural-machine-translation machine-translation visualization attention-visualization
Language:Java 65
youngbin-ro / Multi2OIE
Multi^2OIE: Multilingual Open Information Extraction Based on Multi-Head Attention with BERT (Findings of ACL: EMNLP 2020)
bert information-extraction multi-head-attention multilingualism open-information-extraction sequence-labeling
Language:Python 57
ShaneTian / Att-Induction
Attention-based Induction Networks for Few-Shot Text Classification
few-shot-learning text-classification bert multi-head-attention neural-tensor-network
Language:Python 44
engelnico / point-transformer
This is the official repository of the original Point Transformer architecture.
3d-point-cloud 3d-pointclouds 3d-classification pytorch deep-learning transformer-architecture 3d-segmentation classification segmentation modelnet-dataset modelnet modelnet40 shapenet-dataset shapenetpart transformer attention-mechanism multi-head-attention sortnet
Language:Python 35
JacobHanimann / scDINO
Self-Supervised Vision Transformers for multiplexed imaging datasets
high-content-screening self-supervised-learning single-cell fluorescence-microscopy-imaging greyscale-image morphology multi-channel multi-head-attention phenotyping vision-transformer unsupervised
Language:Python 34
knotgrass / attention
several types of attention modules written in PyTorch
attention attention-mechanism multi-head-attention multi-query-attention pytorch transformer transformers grouped-query-attention scale-dot-product-attention softmax-layer
Language:Python 29
shifop / datagrand_bert
2019达观杯信息提取第5名代码
bert multi-head-attention ner
Language:Python 20
jack57lee / Diversify-MHA
EMNLP 2018: Multi-Head Attention with Disagreement Regularization; NAACL 2019: Information Aggregation for Multi-Head Attention with Routing-by-Agreement
deep-learning multi-head-attention neural-machine-transliteration
Language:Python 18
Zminghua / SentEncoding
Sentence encoder and training code for Mean-Max AAE
representation multi-head-attention
Language:Python 16
M-e-r-c-u-r-y / pytorch-transformers
Collection of different types of transformers for learning purposes
pytorch transformers multi-head-attention multi-query-attention einsum-notation
Language:Jupyter Notebook 14
shreyas-kowshik / nlp4if
Code for the runners up entry on the English subtask on the Shared-Task-On-Fighting the COVID-19 Infodemic, NLP4IF workshop, NAACL'21.
deep-learning multi-head-attention multi-task-learning naacl2021 natural-language-processing
Language:Python 6
pi-tau / transformer
The Transformer model implemented from scratch using PyTorch. The model uses weight sharing between the embedding layers and the pre-softmax linear layer. Training on the Multi30k machine translation task is shown.
deep-learning machine-translation multi-head-attention multi30k pytorch shared-embedding transformer
Language:Python 4
tanishqgautam / Transformers
Pytorch Implementation of Transformers
multi-head-attention transformer nlp self-attention pytorch deep-learning
Language:Python 4
TranQuocTrinh / image_captioning
Image Captioning with Encoder as Efficientnet and Decoder as Decoder of Transformer combined with the attention mechanism.
attention-mechanism convolutional-neural-networks deep-learning efficientnet image-captioning multi-head-attention natural-language-processing python pytorch transformer
Language:Python 4
YigitTurali / HydraViT
HydraViT is a PyTorch implementation of the HydraViT model, an adaptive multi-branch transformer for multi-label disease classification from chest X-ray images. The repository provides the necessary code to train and evaluate the HydraViT model on the NIH Chest X-ray dataset.
chest-xray-images computer-vision deep-learning machine-learning multi-head-attention neural-networks visual-transformers
4
AIMedLab / DeepCE
Code and Datasets for the paper "A deep learning framework for high-throughput mechanism-driven phenotype compound screening and its application to COVID-19 drug repurposing", published on Nature Machine Intelligence in 2021.
phenotypic-drug-discovery graph-neural-network multi-head-attention l1000 covid-19 drug-repurposing chemical-induced-gene-expression
Language:Python 3
gazelle93 / Attention-Various-Positional-Encoding
This project aims to implement the Scaled-Dot-Product Attention layer and the Multi-Head Attention layer using various Positional Encoding methods.
attention-mechanism gensim multi-head-attention natural-language-processing nlp nltk scaled-dot-product spacy wordembeddings pytorch relative-positional-encoding relative-positional-representation t5
Language:Python 2
dev-geof / final-state-transformer
Machine learning development toolkit built upon Transformer encoder network architectures and tailored for the realm of high-energy physics and particle-collision event analysis.
deep-learning machine-learning multi-head-attention particle-physics science-research toolkit transformer
Language:Python 1
liaoyanqing666 / transformer_pytorch
完整的原版transformer程序，complete origin transformer program
beginner multi-head-attention positional-encoding python pytorch transformer
Language:Python 1
navreeetkaur / learn-to-pay-attention
TensorFlow implementation of AlexNet with multi-headed Attention mechanism
tensorflow alexnet attention-mechanism attention-model multi-head-attention
Language:Jupyter Notebook 1
SpydazWebAI-NLP / BasicNeuralNetWork2023
A Basic Multi layered Neural Network, With Attention Masking Features
multi-head-attention neural-network nlp rnn self-attention transformer-architecture
Language:Visual Basic .NET 1
young-zonglin / yangzl-deep-text-matching
Text matching using several deep models.
text-matching deep-model transformer-encoder rnmt-plus-encoder deep-lstms multi-head-attention
Language:Python 1
Transformer_NMT
AshishBodhankar / Transformer_NMT
Attention is all you need: Discovering the Transformer model
attention machine-translation mask multi-head-attention natural-language-processing transformer vaswani
Language:Jupyter Notebook 0
tate8 / translator
Transformer translator website with multithreaded web server in Rust
chatbot css html javascript keras machine-learning multi-head-attention multi-threading positional-encoding python rust tcp tensorflow thread-pool transformer web-server website word-embedding
Language:Rust 0
TmohamedashrafT / vision-transformer-implementation
This repository contains code for implementing Vision Transformer (ViT) model for image classification
multi-head-attention transformers vision-transformer
Language:Python 0
sajith-rahim / transformer-classifier
A Transformer Classifier implemented from Scratch.
classification-model attention multi-head-attention sentence-classification embeddings pytorch scratch-implementation
Language:Python

multi-head-attention

anicolson / DeepXi

sooftware / attentions

imperial-qore / TranAD

poloclub / dodrio

Rintarooo / VRP_DRL_MHA

monk1337 / Various-Attention-mechanisms

datnnt1997 / multi-head_self-attention

zhaocq-nlp / Attention-Visualization

youngbin-ro / Multi2OIE

ShaneTian / Att-Induction

engelnico / point-transformer

JacobHanimann / scDINO

knotgrass / attention

shifop / datagrand_bert

jack57lee / Diversify-MHA

Zminghua / SentEncoding

M-e-r-c-u-r-y / pytorch-transformers

shreyas-kowshik / nlp4if

pi-tau / transformer

tanishqgautam / Transformers

TranQuocTrinh / image_captioning

YigitTurali / HydraViT

AIMedLab / DeepCE

gazelle93 / Attention-Various-Positional-Encoding

dev-geof / final-state-transformer

liaoyanqing666 / transformer_pytorch

navreeetkaur / learn-to-pay-attention

SpydazWebAI-NLP / BasicNeuralNetWork2023

young-zonglin / yangzl-deep-text-matching

AshishBodhankar / Transformer_NMT

tate8 / translator

TmohamedashrafT / vision-transformer-implementation

sajith-rahim / transformer-classifier