multihead-attention

There are 1 repository under multihead-attention topic.

awesome-fast-attention
Separius / awesome-fast-attention
list of efficient attention modules
transformer attention awesome reformer longformer linformer multihead-attention self-attention attention-is-all-you-need transformer-network
Language:Python 978
tlatkowski / multihead-siamese-nets
Implementation of Siamese Neural Networks built upon multihead attention mechanism for text semantic similarity task.
siamese-neural-network multihead-attention semantic-similarity deep-neural-networks siamese-lstm attention deep-architectures deep-learning text-similarity paraphrase paraphrase-identification nlp natural-language-processing quora-question-pairs snli multihead-attention-networks sentence-similarity tensorflow python3 siamese-cnn
Language:Jupyter Notebook 182
datnnt1997 / multi-head_self-attention
A Faster Pytorch Implementation of Multi-Head Self-Attention
attention attention-mechanism multi-head multi-head-attention multi-head-self-attention multihead-attention multihead-self-attention pytorch-self-attention self-attention transformer-attention
Language:Jupyter Notebook 67
tensorops / TransformerX
Flexible Python library providing building blocks (layers) for reproducible Transformers research (Tensorflow ✅, Pytorch 🔜, and Jax 🔜)
attention attention-mechanism deep-learning multihead-attention nlp self-attention transfomers transformers vit xformers
Language:Python 52
jk96491 / Advanced_Models
여러가지 유명한 신경망 모델들을 제공합니다. (DCGAN, VAE, Resnet 등등)
gan resnet-50 vae pytorch dcgan cgan sagan multihead-attention gpt-2
Language:Python 50
akurniawan / pytorch-transformer
Implementation of "Attention is All You Need" paper
attention attention-is-all-you-need multihead-attention pytorch
Language:Python 32
changwookjun / Transformer
Chatbot using Tensorflow (Model is transformer) ko
bert chatbot korean-nlp multihead-attention self-attention tensorflow transformer
Language:Python 28
MirunaPislar / multi-head-attention-labeller
Joint text classification on multiple levels with multiple labels, using a multi-head attention mechanism to wire two prediction tasks together.
attention-mechanism bilstm bilstm-attention conll-2003 error-detection hedge-detection joint-learning joint-models multi-task-learning multihead-attention semi-supervised-learning sentence-classification sentiment-analysis sequence-labelling transformer zero-shot-learning
Language:Python 15
Syeda-Farhat / awesome-Transformers-For-Segmentation
Semantic segmentation is an important job in computer vision, and its applications have grown in popularity over the last decade.We grouped the publications that used various forms of segmentation in this repository. Particularly, every paper is built on a transformer.
computer-vision encoder-decoder instance-segmentation multihead-attention segmentation self-attention semantic-segmentation transformer
15
iafarhan / causal-synthesizer-multihead-attention
Synthesizer Self-Attention is a very recent alternative to causal self-attention that has potential benefits by removing this dot product.
python pytorch attention multihead-attention synthesizer-attention
Language:Python 12
hrithickcodes / transformer-tf
This repository contains the code for the paper "Attention Is All You Need" i.e The Transformer.
attention-is-all-you-need multihead-attention neural-machine-translation self-attention transformer-architecture transformer-tensorflow2 transformers
Language:Jupyter Notebook 8
shawnhan108 / AutoTruckX
An experimental project for autonomous vehicle driving perception with steering angle prediction and semantic segmentation using a combination of UNet, attention and transformers.
autonomous-vehicles autonomous-driving udacity-self-driving-car resnet-50 transfer-learning cnn-lstm conv3d transformer attention semantic-segmentation unet-image-segmentation unet setr multihead-attention steering-angle-prediction
Language:Python 7
antonio-f / GPT_from_scratch
Very simple implementation of GPT architecture using PyTorch and Jupyter.
easy-to-use gpt jupyter-notebook python pytorch tutorial from-scratch noob-friendly simple newbie transformer multihead-attention self-attention triangular-matrix bigram-model
Language:Jupyter Notebook 5
bkhanal-11 / transformers
The implementation of transformer as presented in the paper "Attention is all you need" from scratch.
attention-is-all-you-need attention-mechanism multihead-attention positional-encoding self-attention transformers
Language:Python 4
abhilash1910 / GraphAttentionNetworks
This package is a Tensorflow2/Keras implementation for Graph Attention Network embeddings and also provides a Trainable layer for Multihead Graph Attention.
tf2 graph-attention-networks multihead-attention self-attention leaky-relu keras-tensorflow
Language:Python 3
jaydeepthik / Nano-GPT
Simple GPT with multiheaded attention for char level tokens, inspired from Andrej Karpathy's video lectures : https://github.com/karpathy/ng-video-lecture
gpt multihead-attention pytorch pytorch-implementation transformers
Language:Jupyter Notebook 3
achiverram28 / FedLSF-DCOSS
Official implementation of the paper "FedLSF: Federated Local Graph Learning via Specformers"
eigenvalues eigenvectors federated-learning graphneuralnetwork graphs multihead-attention spectral-gnns
Language:Python 1
dcarpintero / transformer101
Annotated vanilla implementation in PyTorch of the Transformer model introduced in 'Attention Is All You Need'
attention-is-all-you-need encoder-decoder-architecture multihead-attention positional-encoding self-attention transfomer dot-product-attention dropout-layers feedforward-neural-network gelu linear-layers normalization-layers softmax pytorch
Language:Jupyter Notebook 1
mpalaourg / PGL-SUM
A PyTorch Implementation of PGL-SUM from "Combining Global and Local Attention with Positional Encoding for Video Summarization", Proc. IEEE ISM 2021
video-summarization supervised-learning computer-vision deep-learning self-attention multihead-attention positional-encoding ism21
Language:Python 1
whsqkaak / attentions_pytorch
A repository for implementations of attention mechanism by PyTorch.
attention attention-mechanism dot-product-attention multihead-attention pytorch scaled-dot-product-attention
Language:Python 1
Mascerade / scale-transformer-encoder
A Transformer Encoder where the embedding size can be down-sized.
transformer encoder nlp ai ml machine learning machine-learning artificial-intelligence multihead-attention sequence
Language:Python 0
OscarHChung / GPT-Model
GPT model that can take a text file from anywhere on the internet and imitate the linguistic style of the text
bigram-model gpt machine-learning multihead-attention python
Language:Python 0
puskal-khadka / Transformer
Transformer model based on the research paper: "𝗔𝘁𝘁𝗲𝗻𝘁𝗶𝗼𝗻 𝗜𝘀 𝗔𝗹𝗹 𝗬𝗼𝘂 𝗡𝗲𝗲𝗱"
attention-is-all-you-need deep-neural-networks multihead-attention pytorch seq2seq transformer transformermodel
Language:Python 0
Resh-97 / MixSeq-Connecting-Macroscopic-Time-Series-Forecasting-with-Microscopic-Time-Series-Data
Testing the Reproducibility of the paper: MixSeq. Under the assumption that macroscopic time series follow a mixture distribution, they hypothesise that lower variance of constituting latent mixture components could improve the estimation of macroscopic time series.
arma deepar multihead-attention time-series vae-implementation vae-pytorch reproducibility-challenge comp6248
Language:Jupyter Notebook 0
sarthak7509 / ConversationalAi
This is implementation of famous multi head attention mode for conversational ai paper. This model is trained on both Cornell movie data set and WikkiQna data set provided by microsoft
transformer multihead-attention tensorflow python3 self-attention neural-networks
Language:Jupyter Notebook 0
vasisthasinghal / Machine_Translation
Machine Translation models (with and without attention) to convert sentences in Tamil to Hindi. Transformer models are also used for this same task and performance is compared.
tokenization machine-translation transformer attention-mechanism multihead-attention deeplearning pytorch pytorch-implementation
Language:Jupyter Notebook 0
aman-17 / 3dprinting-extrusion-detection
3D Printing Extrusion Detection using Multi-Head Attention Model
3d-printing deep-learning multihead-attention python
Language:Python
Group-1-ET / English-Telugu-Translator
Deployed locally
css encoder-decoder flask html multihead-attention transformers
Language:Python
meme2515 / transformer_pytorch
PyTorch implementation of the Transformer architecture from the paper Attention is All You Need. Includes implementation of attention mechanism.
transformer attention multihead-attention pytorch bert gpt
Language:Python
varunram2001 / MHA-Module-for-Attention-based-Deep-Learning
This repository contains the code for a Multi Scale attention based module that was built and tested on a data set containing Concrete crack images. It was later tested with other data sets as well. Provided a better accuracy compared to the standard approach.
attention-mechanism deep-learning deep-neural-networks multihead-attention multihead-attention-networks
Language:Jupyter Notebook
yl-jiang / Transformer
Attention is all you need with Pytorch
attention-mechanism multihead-attention python3 pytorch-implementation transformer translation
Language:Jupyter Notebook

multihead-attention

Separius / awesome-fast-attention

tlatkowski / multihead-siamese-nets

datnnt1997 / multi-head_self-attention

tensorops / TransformerX

jk96491 / Advanced_Models

akurniawan / pytorch-transformer

changwookjun / Transformer

MirunaPislar / multi-head-attention-labeller

Syeda-Farhat / awesome-Transformers-For-Segmentation

iafarhan / causal-synthesizer-multihead-attention

hrithickcodes / transformer-tf

shawnhan108 / AutoTruckX

antonio-f / GPT_from_scratch

bkhanal-11 / transformers

abhilash1910 / GraphAttentionNetworks

jaydeepthik / Nano-GPT

achiverram28 / FedLSF-DCOSS

dcarpintero / transformer101

mpalaourg / PGL-SUM

whsqkaak / attentions_pytorch

Mascerade / scale-transformer-encoder

OscarHChung / GPT-Model

puskal-khadka / Transformer

Resh-97 / MixSeq-Connecting-Macroscopic-Time-Series-Forecasting-with-Microscopic-Time-Series-Data

sarthak7509 / ConversationalAi

vasisthasinghal / Machine_Translation

aman-17 / 3dprinting-extrusion-detection

Group-1-ET / English-Telugu-Translator

meme2515 / transformer_pytorch

varunram2001 / MHA-Module-for-Attention-based-Deep-Learning

yl-jiang / Transformer