multihead-attention

There are 1 repository under multihead-attention topic.

awesome-fast-attention
Separius / awesome-fast-attention
list of efficient attention modules
transformer attention awesome reformer longformer linformer multihead-attention self-attention attention-is-all-you-need transformer-network
Language:Python 990
tlatkowski / multihead-siamese-nets
Implementation of Siamese Neural Networks built upon multihead attention mechanism for text semantic similarity task.
attention deep-architectures deep-learning deep-neural-networks multihead-attention multihead-attention-networks natural-language-processing nlp paraphrase paraphrase-identification python3 quora-question-pairs semantic-similarity sentence-similarity siamese-cnn siamese-lstm siamese-neural-network snli tensorflow text-similarity
Language:Jupyter Notebook 183
datnnt1997 / multi-head_self-attention
A Faster Pytorch Implementation of Multi-Head Self-Attention
attention attention-mechanism multi-head multi-head-attention multi-head-self-attention multihead-attention multihead-self-attention pytorch-self-attention self-attention transformer-attention
Language:Jupyter Notebook 71
tensorops / TransformerX
Flexible Python library providing building blocks (layers) for reproducible Transformers research (Tensorflow ✅, Pytorch 🔜, and Jax 🔜)
attention attention-mechanism deep-learning transfomers vit xformers multihead-attention nlp self-attention transformers
Language:Python 51
jk96491 / Advanced_Models
여러가지 유명한 신경망 모델들을 제공합니다. (DCGAN, VAE, Resnet 등등)
cgan dcgan gan gpt-2 multihead-attention pytorch resnet-50 sagan vae
Language:Python 50
akurniawan / pytorch-transformer
Implementation of "Attention is All You Need" paper
pytorch attention attention-is-all-you-need multihead-attention
Language:Python 32
changwookjun / Transformer
Chatbot using Tensorflow (Model is transformer) ko
bert chatbot korean-nlp multihead-attention self-attention tensorflow transformer
Language:Python 29
Syeda-Farhat / awesome-Transformers-For-Segmentation
Semantic segmentation is an important job in computer vision, and its applications have grown in popularity over the last decade.We grouped the publications that used various forms of segmentation in this repository. Particularly, every paper is built on a transformer.
computer-vision encoder-decoder instance-segmentation multihead-attention segmentation self-attention semantic-segmentation transformer
27
MirunaPislar / multi-head-attention-labeller
Joint text classification on multiple levels with multiple labels, using a multi-head attention mechanism to wire two prediction tasks together.
sequence-labelling sentence-classification joint-models joint-learning multi-task-learning multihead-attention transformer attention-mechanism bilstm-attention bilstm zero-shot-learning semi-supervised-learning conll-2003 error-detection sentiment-analysis hedge-detection
Language:Python 16
iafarhan / causal-synthesizer-multihead-attention
Synthesizer Self-Attention is a very recent alternative to causal self-attention that has potential benefits by removing this dot product.
attention multihead-attention python pytorch synthesizer-attention
Language:Python 12
hrithickcodes / transformer-tf
This repository contains the code for the paper "Attention Is All You Need" i.e The Transformer.
attention-is-all-you-need multihead-attention neural-machine-translation self-attention transformer-architecture transformer-tensorflow2 transformers
Language:Jupyter Notebook 8
shawnhan108 / AutoTruckX
An experimental project for autonomous vehicle driving perception with steering angle prediction and semantic segmentation using a combination of UNet, attention and transformers.
autonomous-vehicles autonomous-driving udacity-self-driving-car resnet-50 transfer-learning cnn-lstm conv3d transformer attention semantic-segmentation unet-image-segmentation unet setr multihead-attention steering-angle-prediction
Language:Python 7
bkhanal-11 / transformers
The implementation of transformer as presented in the paper "Attention is all you need" from scratch.
attention-is-all-you-need attention-mechanism multihead-attention positional-encoding self-attention transformers
Language:Python 6
jaydeepthik / Nano-GPT
Simple GPT with multiheaded attention for char level tokens, inspired from Andrej Karpathy's video lectures : https://github.com/karpathy/ng-video-lecture
gpt pytorch-implementation multihead-attention pytorch transformers
Language:Jupyter Notebook 5
antonio-f / GPT_from_scratch
Very simple implementation of GPT architecture using PyTorch and Jupyter.
bigram-model easy-to-use from-scratch gpt jupyter-notebook multihead-attention newbie noob-friendly python pytorch self-attention simple transformer triangular-matrix tutorial
Language:Jupyter Notebook 4
abhilash1910 / GraphAttentionNetworks
This package is a Tensorflow2/Keras implementation for Graph Attention Network embeddings and also provides a Trainable layer for Multihead Graph Attention.
graph-attention-networks keras-tensorflow leaky-relu multihead-attention self-attention tf2
Language:Python 3
mpalaourg / PGL-SUM
A PyTorch Implementation of PGL-SUM from "Combining Global and Local Attention with Positional Encoding for Video Summarization", Proc. IEEE ISM 2021
computer-vision deep-learning ism21 multihead-attention positional-encoding self-attention supervised-learning video-summarization
Language:Python 2
achiverram28 / FedLSF-DCOSS
Official implementation of the paper "FedLSF: Federated Local Graph Learning via Specformers"
eigenvalues eigenvectors federated-learning graphneuralnetwork graphs multihead-attention spectral-gnns
Language:Python 1
dcarpintero / transformer101
Annotated vanilla implementation in PyTorch of the Transformer model introduced in 'Attention Is All You Need'.
attention-is-all-you-need encoder-decoder-architecture multihead-attention positional-encoding self-attention transfomer dot-product-attention dropout-layers feedforward-neural-network gelu linear-layers normalization-layers softmax pytorch
Language:Jupyter Notebook 1
puskal-khadka / Transformer
Transformer model based on the research paper: "𝗔𝘁𝘁𝗲𝗻𝘁𝗶𝗼𝗻 𝗜𝘀 𝗔𝗹𝗹 𝗬𝗼𝘂 𝗡𝗲𝗲𝗱"
attention-is-all-you-need deep-neural-networks multihead-attention pytorch seq2seq transformer transformermodel
Language:Python 1
Resh-97 / MixSeq-Connecting-Macroscopic-Time-Series-Forecasting-with-Microscopic-Time-Series-Data
Testing the Reproducibility of the paper: MixSeq. Under the assumption that macroscopic time series follow a mixture distribution, they hypothesise that lower variance of constituting latent mixture components could improve the estimation of macroscopic time series.
arma deepar multihead-attention time-series vae-implementation vae-pytorch reproducibility-challenge comp6248
Language:Jupyter Notebook 1
whsqkaak / attentions_pytorch
A repository for implementations of attention mechanism by PyTorch.
attention attention-mechanism dot-product-attention multihead-attention pytorch scaled-dot-product-attention
Language:Python 1
Group-1-ET / English-Telugu-Translator
Deployed locally
css encoder-decoder flask html multihead-attention transformers
Language:Python 0
Mascerade / scale-transformer-encoder
A Transformer Encoder where the embedding size can be down-sized.
ai artificial-intelligence encoder learning machine machine-learning ml multihead-attention nlp sequence transformer
Language:Python 0
OscarHChung / Slang-Emulator
At its core, a GPT model that can take a text file from anywhere on the internet or from local files and imitate the linguistic style of the text
bigram-model gpt machine-learning multihead-attention python
Language:Python 0
Pranavhc / Shakespearean-Text-Generator
A Decoder-only Transfomer model for text generation.
attention-mechanism multihead-attention natural-language-processing neural-network transformer
Language:Jupyter Notebook 0
sarthak7509 / ConversationalAi
This is implementation of famous multi head attention mode for conversational ai paper. This model is trained on both Cornell movie data set and WikkiQna data set provided by microsoft
transformer multihead-attention tensorflow python3 self-attention neural-networks
Language:Jupyter Notebook 0
vasisthasinghal / Machine_Translation
Machine Translation models (with and without attention) to convert sentences in Tamil to Hindi. Transformer models are also used for this same task and performance is compared.
tokenization machine-translation transformer attention-mechanism multihead-attention deeplearning pytorch pytorch-implementation
Language:Jupyter Notebook 0
aman-17 / 3dprinting-extrusion-detection
3D Printing Extrusion Detection using Multi-Head Attention Model
3d-printing deep-learning multihead-attention python
Language:Python
aniketDash7 / multihead_attention_implementation
Implementation of Multihead attention mechanism using numpy and pyTorch
multihead-attention numpy pytorch torch
Language:Jupyter Notebook
JivanAcharya / Shakespeare-GPT
Implementing a GPT (Generative Pre-trained Transformer) model from scratch on Shakespeare's work.
gpt multihead-attention self-attention transformer
Language:Jupyter Notebook
meme2515 / transformer_pytorch
PyTorch implementation of the Transformer architecture from the paper Attention is All You Need. Includes implementation of attention mechanism.
transformer attention multihead-attention pytorch bert gpt
Language:Python
varunram2001 / MHA-Module-for-Attention-based-Deep-Learning
This repository contains the code for a Multi Scale attention based module that was built and tested on a data set containing Concrete crack images. It was later tested with other data sets as well. Provided a better accuracy compared to the standard approach.
attention-mechanism deep-learning deep-neural-networks multihead-attention multihead-attention-networks
Language:Jupyter Notebook
yl-jiang / Transformer
Attention is all you need with Pytorch
attention-mechanism multihead-attention python3 pytorch-implementation transformer translation
Language:Jupyter Notebook

multihead-attention

Separius / awesome-fast-attention

tlatkowski / multihead-siamese-nets

datnnt1997 / multi-head_self-attention

tensorops / TransformerX

jk96491 / Advanced_Models

akurniawan / pytorch-transformer

changwookjun / Transformer

Syeda-Farhat / awesome-Transformers-For-Segmentation

MirunaPislar / multi-head-attention-labeller

iafarhan / causal-synthesizer-multihead-attention

hrithickcodes / transformer-tf

shawnhan108 / AutoTruckX

bkhanal-11 / transformers

jaydeepthik / Nano-GPT

antonio-f / GPT_from_scratch

abhilash1910 / GraphAttentionNetworks

mpalaourg / PGL-SUM

achiverram28 / FedLSF-DCOSS

dcarpintero / transformer101

puskal-khadka / Transformer

Resh-97 / MixSeq-Connecting-Macroscopic-Time-Series-Forecasting-with-Microscopic-Time-Series-Data

whsqkaak / attentions_pytorch

Group-1-ET / English-Telugu-Translator

Mascerade / scale-transformer-encoder

OscarHChung / Slang-Emulator

Pranavhc / Shakespearean-Text-Generator

sarthak7509 / ConversationalAi

vasisthasinghal / Machine_Translation

aman-17 / 3dprinting-extrusion-detection

aniketDash7 / multihead_attention_implementation

JivanAcharya / Shakespeare-GPT

meme2515 / transformer_pytorch

varunram2001 / MHA-Module-for-Attention-based-Deep-Learning

yl-jiang / Transformer