Phil Wang's repositories
vit-pytorch
Implementation of Vision Transformer, a simple way to achieve SOTA in vision classification with only a single transformer encoder, in Pytorch
DALLE2-pytorch
Implementation of DALL-E 2, OpenAI's updated text-to-image synthesis neural network, in Pytorch
denoising-diffusion-pytorch
Implementation of Denoising Diffusion Probabilistic Model in Pytorch
x-transformers
A simple but complete full-attention transformer with a set of promising experimental features from various papers
vector-quantize-pytorch
Vector (and Scalar) Quantization, in Pytorch
lion-pytorch
🦁 Lion, new optimizer discovered by Google Brain using genetic algorithms that is purportedly better than Adam(w), in Pytorch
byol-pytorch
Usable Implementation of "Bootstrap Your Own Latent" self-supervised learning, from Deepmind, in Pytorch
gigagan-pytorch
Implementation of GigaGAN, new SOTA GAN out of Adobe. Culmination of nearly a decade of research into GANs
linear-attention-transformer
Transformer based on a variant of attention that is linear complexity in respect to sequence length
alphafold3-pytorch
Implementation of Alphafold 3 in Pytorch
meshgpt-pytorch
Implementation of MeshGPT, SOTA Mesh generation using Attention, in Pytorch
magvit2-pytorch
Implementation of MagViT2 Tokenizer in Pytorch
rotary-embedding-torch
Implementation of Rotary Embeddings, from the Roformer paper, in Pytorch
ema-pytorch
A simple way to keep track of an Exponential Moving Average (EMA) version of your pytorch model
iTransformer
Unofficial implementation of iTransformer - SOTA Time Series Forecasting using Attention networks, out of Tsinghua / Ant group
BS-RoFormer
Implementation of Band Split Roformer, SOTA Attention network for music source separation out of ByteDance AI Labs
q-transformer
Implementation of Q-Transformer, Scalable Offline Reinforcement Learning via Autoregressive Q-Functions, out of Google Deepmind
classifier-free-guidance-pytorch
Implementation of Classifier Free Guidance in Pytorch, with emphasis on text conditioning, and flexibility to include multiple text embedding models
st-moe-pytorch
Implementation of ST-Moe, the latest incarnation of MoE after years of research at Brain, in Pytorch
lumiere-pytorch
Implementation of Lumiere, SOTA text-to-video generation from Google Deepmind, in Pytorch
En-transformer
Implementation of E(n)-Transformer, which incorporates attention mechanisms into Welling's E(n)-Equivariant Graph Neural Network
bidirectional-cross-attention
A simple cross attention that updates both the source and target in one step
pytorch-custom-utils
Just some miscellaneous utility functions / decorators / modules related to Pytorch and Accelerate to help speed up implementation of new AI research
infini-transformer-pytorch
Implementation of Infini-Transformer in Pytorch
self-reasoning-tokens-pytorch
Exploration into the proposed "Self Reasoning Tokens" by Felipe Bonetto
frame-averaging-pytorch
Pytorch implementation of a simple way to enable (Stochastic) Frame Averaging for any network
nim-genetic-algorithm
a simple genetic algorithm written in Nim
crystal-genetic-algorithm
A simple toy genetic algorithm in Crystal