Phil Wang's repositories
vit-pytorch
Implementation of Vision Transformer, a simple way to achieve SOTA in vision classification with only a single transformer encoder, in Pytorch
denoising-diffusion-pytorch
Implementation of Denoising Diffusion Probabilistic Model in Pytorch
x-transformers
A simple but complete full-attention transformer with a set of promising experimental features from various papers
vector-quantize-pytorch
Vector (and Scalar) Quantization, in Pytorch
lion-pytorch
🦁 Lion, new optimizer discovered by Google Brain using genetic algorithms that is purportedly better than Adam(w), in Pytorch
gigagan-pytorch
Implementation of GigaGAN, new SOTA GAN out of Adobe. Culmination of nearly a decade of research into GANs
byol-pytorch
Usable Implementation of "Bootstrap Your Own Latent" self-supervised learning, from Deepmind, in Pytorch
alphafold3-pytorch
Implementation of Alphafold 3 in Pytorch
meshgpt-pytorch
Implementation of MeshGPT, SOTA Mesh generation using Attention, in Pytorch
magvit2-pytorch
Implementation of MagViT2 Tokenizer in Pytorch
rotary-embedding-torch
Implementation of Rotary Embeddings, from the Roformer paper, in Pytorch
ema-pytorch
A simple way to keep track of an Exponential Moving Average (EMA) version of your pytorch model
ring-attention-pytorch
Implementation of 💍 Ring Attention, from Liu et al. at Berkeley AI, in Pytorch
local-attention
An implementation of local windowed attention for language modeling
classifier-free-guidance-pytorch
Implementation of Classifier Free Guidance in Pytorch, with emphasis on text conditioning, and flexibility to include multiple text embedding models
st-moe-pytorch
Implementation of ST-Moe, the latest incarnation of MoE after years of research at Brain, in Pytorch
CoLT5-attention
Implementation of the conditionally routed attention in the CoLT5 architecture, in Pytorch
e2-tts-pytorch
Implementation of E2-TTS, "Embarrassingly Easy Fully Non-Autoregressive Zero-Shot TTS", in Pytorch
titok-pytorch
Implementation of TiTok, proposed by Bytedance in "An Image is Worth 32 Tokens for Reconstruction and Generation"
pytorch-custom-utils
Just some miscellaneous utility functions / decorators / modules related to Pytorch and Accelerate to help speed up implementation of new AI research
multimodal-dit-pytorch
Implementation of a multimodal diffusion transformer in Pytorch
rectified-flow-pytorch
Implementation of rectified flow and some of its followup research / improvements in Pytorch
gateloop-transformer
Implementation of GateLoop Transformer in Pytorch and Jax
agent-attention-pytorch
Implementation of Agent Attention in Pytorch
diffusion-policy
Implementation of Diffusion Policy, Toyota Research's supposed breakthrough in leveraging DDPMs for learning policies for real-world Robotics
grokfast-pytorch
Explorations into the proposal from the paper "Grokfast, Accelerated Grokking by Amplifying Slow Gradients"
PEER-pytorch
Pytorch implementation of the PEER block from the paper, Mixture of A Million Experts, by Xu Owen He at Deepmind
flash-attention
Fast and memory-efficient exact attention