attention-mechanisms

There are 5 repositories under attention-mechanisms topic.

lucidrains / PaLM-rlhf-pytorch
Implementation of RLHF (Reinforcement Learning with Human Feedback) on top of the PaLM architecture. Basically ChatGPT but with PaLM
artificial-intelligence attention-mechanisms deep-learning human-feedback reinforcement-learning transformers
Language:Python 7867
cmhungsteve / Awesome-Transformer-Attention
An ultimately comprehensive paper list of Vision Transformer/Attention, including papers, codes, and related websites
attention-mechanism attention-mechanisms awesome-list computer-vision deep-learning detr papers self-attention transformer transformer-architecture transformer-awesome transformer-cv transformer-models transformer-with-cv transformers vision-transformer visual-transformer vit
4925
lucidrains / musiclm-pytorch
Implementation of MusicLM, Google's new SOTA model for music generation using attention networks, in Pytorch
artificial-intelligence attention-mechanisms deep-learning music-synthesis transformers
Language:Python 3274
lucidrains / audiolm-pytorch
Implementation of AudioLM, a SOTA Language Modeling Approach to Audio Generation out of Google Research, in Pytorch
artificial-intelligence attention-mechanisms audio-synthesis deep-learning transformers
Language:Python 2587
lucidrains / toolformer-pytorch
Implementation of Toolformer, Language Models That Can Use Tools, by MetaAI
api-calling artificial-intelligence attention-mechanisms deep-learning transformers
Language:Python 2050
lucidrains / make-a-video-pytorch
Implementation of Make-A-Video, new SOTA text to video generator from Meta AI, in Pytorch
artificial-intelligence attention-mechanisms axial-convolutions deep-learning text-to-video
Language:Python 1980
lucidrains / alphafold3-pytorch
Implementation of Alphafold 3 from Google Deepmind in Pytorch
artificial-intelligence attention-mechanisms deep-learning denoising-diffusion protein-structure-prediction transformers
Language:Python 1527
pprp / awesome-attention-mechanism-in-cv
Awesome List of Attention Modules and Plug&Play Modules in Computer Vision
pytorch-attention attention-model attention-mechanisms implementation vision-transformer plugandplay computer-vision
Language:Python 1218
lucidrains / muse-maskgit-pytorch
Implementation of Muse: Text-to-Image Generation via Masked Generative Transformers, in Pytorch
artificial-intelligence deep-learning text-to-image attention-mechanisms transformers
Language:Python 910
lucidrains / meshgpt-pytorch
Implementation of MeshGPT, SOTA Mesh generation using Attention, in Pytorch
artificial-intelligence attention-mechanisms deep-learning mesh-generation transformers
Language:Python 849
lucidrains / phenaki-pytorch
Implementation of Phenaki Video, which uses Mask GIT to produce text guided videos of up to 2 minutes in length, in Pytorch
artificial-intelligence attention-mechanisms deep-learning text-to-video transformers imagination-machine
Language:Python 780
kyegomez / LongNet
Implementation of plug in and play Attention from "LongNet: Scaling Transformers to 1,000,000,000 Tokens"
artificial-intelligence attention attention-is-all-you-need attention-mechanisms chatgpt gpt3 gpt4 machine-learning transformer context-length
Language:Python 711
JulesBelveze / time-series-autoencoder
PyTorch Dual-Attention LSTM-Autoencoder For Multivariate Time Series
autoencoder lstm-autoencoder attention-mechanisms time-series multivariate-timeseries pytorch forecasting
Language:Python 687
lucidrains / MEGABYTE-pytorch
Implementation of MEGABYTE, Predicting Million-byte Sequences with Multiscale Transformers, in Pytorch
artificial-intelligence deep-learning learned-tokenization attention-mechanisms long-context transformers
Language:Python 651
lucidrains / magvit2-pytorch
Implementation of MagViT2 Tokenizer in Pytorch
artificial-intelligence attention-mechanisms deep-learning finite-scalar-quantization transformers video-generation
Language:Python 632
lucidrains / BS-RoFormer
Implementation of Band Split Roformer, SOTA Attention network for music source separation out of ByteDance AI Labs
artificial-intelligence attention-mechanisms deep-learning music-source-separation transformers
Language:Python 616
lucidrains / iTransformer
Unofficial implementation of iTransformer - SOTA Time Series Forecasting using Attention networks, out of Tsinghua / Ant group
artificial-intelligence attention-mechanisms deep-learning time-series-forecasting transformers
Language:Python 515
changzy00 / pytorch-attention
🦖Pytorch implementation of popular Attention Mechanisms, Vision Transformers, MLP-Like models and CNNs.🔥🔥🔥
attention-mechanisms cnns mlp-networks pytorch-implementation vits
Language:Python 505
lucidrains / local-attention
An implementation of local windowed attention for language modeling
artificial-intelligence attention-mechanisms deep-learning
Language:Python 475
lucidrains / mmdit
Implementation of a single layer of the MMDiT, proposed in Stable Diffusion 3, in Pytorch
artificial-intelligence attention-mechanisms deep-learning multi-modal-attention
Language:Python 442
lucidrains / robotic-transformer-pytorch
Implementation of RT1 (Robotic Transformer) in Pytorch
artificial-intelligence attention-mechanisms deep-learning robotics transformers
Language:Python 441
triplet-attention
landskape-ai / triplet-attention
Official PyTorch Implementation for "Rotate to Attend: Convolutional Triplet Attention Module." [WACV 2021]
arxiv attention-mechanism attention-mechanisms computer-vision convolutional-neural-networks deep-learning detection gradcam imagenet paper triplet-attention
Language:Jupyter Notebook 430
lucidrains / recurrent-memory-transformer-pytorch
Implementation of Recurrent Memory Transformer, Neurips 2022 paper, in Pytorch
artificial-intelligence attention-mechanisms deep-learning transformers long-context memory recurrence
Language:Python 414
lucidrains / q-transformer
Implementation of Q-Transformer, Scalable Offline Reinforcement Learning via Autoregressive Q-Functions, out of Google Deepmind
artificial-intelligence attention-mechanisms deep-learning offline-learning q-learning robotics transformers
Language:Python 397
lucidrains / clinical-calculator-tooluse
Explorations into training LLMs to use clinical calculators from patient history, using open sourced models. Will start with Wells' Criteria
artificial-intelligence attention-mechanisms clinical-calculators deep-learning medicine transformers
Language:Python 315
lucidrains / equiformer-pytorch
Implementation of the Equiformer, SE3/E3 equivariant attention network that reaches new SOTA, and adopted for use by EquiFold for protein folding
artificial-intelligence deep-learning equivariance transformers attention-mechanisms protein-folding molecules
Language:Python 274
cbaziotis / neat-vision
Neat (Neural Attention) Vision, is a visualization tool for the attention mechanisms of deep-learning models for Natural Language Processing (NLP) tasks. (framework-agnostic)
attention attention-mechanism attention-visualization attention-scores text-visualization deep-learning deep-learning-visualization vuejs nlp natural-language-processing attention-mechanisms visualization deep-learning-library self-attention self-attentive-rnn
Language:Vue 251
lucidrains / CoLT5-attention
Implementation of the conditionally routed attention in the CoLT5 architecture, in Pytorch
artificial-intelligence attention-mechanisms deep-learning efficient-attention routing
Language:Python 229
vene / sparse-structured-attention
Sparse and structured neural attention mechanisms
attention-mechanism attention-mechanisms deep-learning deep-neural-networks deeplearning fused-lasso segmentation sparse sparsity
Language:Python 224
lucidrains / simple-hierarchical-transformer
Experiments around a simple idea for inducing multiple hierarchical predictive model within a GPT
artificial-intelligence deep-learning attention-mechanisms transformers hierarchical-predictive-coding
Language:Python 221
lucidrains / block-recurrent-transformer-pytorch
Implementation of Block Recurrent Transformer - Pytorch
artificial-intelligence attention-mechanisms deep-learning long-context-attention long-context-transformers memory recurrence
Language:Python 220
GiantPandaCV / yolov3-point
Learning YOLOv3 from scratch 从零开始学习YOLOv3代码
yolov3 attention-mechanism attention-mechanisms plug-and-play object-detection
Language:Jupyter Notebook 216
lucidrains / flash-attention-jax
Implementation of Flash Attention in Jax
artificial-intelligence deep-learning attention-mechanisms jax long-context-attention
Language:Python 216
lucidrains / flash-cosine-sim-attention
Implementation of fused cosine similarity attention in the same style as Flash Attention
artificial-intelligence attention-mechanisms deep-learning
Language:Cuda 214
kyegomez / MambaTransformer
Integrating Mamba/SSMs with Transformer for Enhanced Long Context and High-Quality Sequence Modeling
ai artificial-intelligence attention-is-all-you-need attention-mechanisms gpt4 language machine-learning multimodal neural-network neural-networks pytorch recurrent-neural-networks rnns ssm tensorflow zeta
Language:Python 206
lucidrains / recurrent-interface-network-pytorch
Implementation of Recurrent Interface Network (RIN), for highly efficient generation of images and video without cascading networks, in Pytorch
artificial-intelligence attention-mechanisms deep-learning denoising-diffusion image-generation latents video-generation
Language:Python 206

attention-mechanisms

lucidrains / PaLM-rlhf-pytorch

cmhungsteve / Awesome-Transformer-Attention

lucidrains / musiclm-pytorch

lucidrains / audiolm-pytorch

lucidrains / toolformer-pytorch

lucidrains / make-a-video-pytorch

lucidrains / alphafold3-pytorch

pprp / awesome-attention-mechanism-in-cv

lucidrains / muse-maskgit-pytorch

lucidrains / meshgpt-pytorch

lucidrains / phenaki-pytorch

kyegomez / LongNet

JulesBelveze / time-series-autoencoder

lucidrains / MEGABYTE-pytorch

lucidrains / magvit2-pytorch

lucidrains / BS-RoFormer

lucidrains / iTransformer

changzy00 / pytorch-attention

lucidrains / local-attention

lucidrains / mmdit

lucidrains / robotic-transformer-pytorch

landskape-ai / triplet-attention

lucidrains / recurrent-memory-transformer-pytorch

lucidrains / q-transformer

lucidrains / clinical-calculator-tooluse

lucidrains / equiformer-pytorch

cbaziotis / neat-vision

lucidrains / CoLT5-attention

vene / sparse-structured-attention

lucidrains / simple-hierarchical-transformer

lucidrains / block-recurrent-transformer-pytorch

GiantPandaCV / yolov3-point

lucidrains / flash-attention-jax

lucidrains / flash-cosine-sim-attention

kyegomez / MambaTransformer

lucidrains / recurrent-interface-network-pytorch