Phil Wang's repositories
vit-pytorch
Implementation of Vision Transformer, a simple way to achieve SOTA in vision classification with only a single transformer encoder, in Pytorch
denoising-diffusion-pytorch
Implementation of Denoising Diffusion Probabilistic Model in Pytorch
imagen-pytorch
Implementation of Imagen, Google's Text-to-Image Neural Network, in Pytorch
x-transformers
A concise but complete full-attention transformer with a set of promising experimental features from various papers
vector-quantize-pytorch
Vector (and Scalar) Quantization, in Pytorch
audiolm-pytorch
Implementation of AudioLM, a SOTA Language Modeling Approach to Audio Generation out of Google Research, in Pytorch
alphafold3-pytorch
Implementation of Alphafold 3 in Pytorch
meshgpt-pytorch
Implementation of MeshGPT, SOTA Mesh generation using Attention, in Pytorch
transfusion-pytorch
Pytorch implementation of Transfusion, "Predict the Next Token and Diffuse Images with One Multi-Modal Model", from MetaAI
magvit2-pytorch
Implementation of MagViT2 Tokenizer in Pytorch
ema-pytorch
A simple way to keep track of an Exponential Moving Average (EMA) version of your Pytorch model
ring-attention-pytorch
Implementation of 💍 Ring Attention, from Liu et al. at Berkeley AI, in Pytorch
enformer-pytorch
Implementation of Enformer, Deepmind's attention network for predicting gene expression, in Pytorch
robotic-transformer-pytorch
Implementation of RT1 (Robotic Transformer) in Pytorch
classifier-free-guidance-pytorch
Implementation of Classifier Free Guidance in Pytorch, with emphasis on text conditioning, and flexibility to include multiple text embedding models
q-transformer
Implementation of Q-Transformer, Scalable Offline Reinforcement Learning via Autoregressive Q-Functions, out of Google Deepmind
e2-tts-pytorch
Implementation of E2-TTS, "Embarrassingly Easy Fully Non-Autoregressive Zero-Shot TTS", in Pytorch
minGRU-pytorch
Implementation of the proposed minGRU in Pytorch
nGPT-pytorch
Quick implementation of nGPT, learning entirely on the hypersphere, from NvidiaAI
mixture-of-attention
Some personal experiments around routing tokens to different autoregressive attention, akin to mixture-of-experts
infini-transformer-pytorch
Implementation of Infini-Transformer in Pytorch
adam-atan2-pytorch
Implementation of the proposed Adam-atan2 from Google Deepmind in Pytorch
rvq-vae-gpt
My attempts at applying Soundstream design on learned tokenization of text and then applying hierarchical attention to text generation
spline-based-transformer
Implementation of the proposed Spline-Based Transformer from Disney Research
maskbit-pytorch
Implementation of the proposed MaskBit from Bytedance AI
lvsm-pytorch
Implementation of LVSM, Large View Synthesis with Minimal 3d Inductive Bias
phasic-policy-gradient
An implementation of Phasic Policy Gradient, a proposed improvement of Proximal Policy Gradients, in Pytorch
SAC-pytorch
Implementation of Soft Actor Critic and some of its improvements in Pytorch
vit-arc-slot
Explorations into improving ViTArc with Slot Attention