Phil Wang's repositories
DALLE2-pytorch
Implementation of DALL-E 2, OpenAI's updated text-to-image synthesis neural network, in Pytorch
PaLM-rlhf-pytorch
Implementation of RLHF (Reinforcement Learning with Human Feedback) on top of the PaLM architecture. Basically ChatGPT but with PaLM
audiolm-pytorch
Implementation of AudioLM, a SOTA Language Modeling Approach to Audio Generation out of Google Research, in Pytorch
toolformer-pytorch
Implementation of Toolformer, Language Models That Can Use Tools, by MetaAI
CoCa-pytorch
Implementation of CoCa, Contrastive Captioners are Image-Text Foundation Models, in Pytorch
tab-transformer-pytorch
Implementation of TabTransformer, attention network for tabular data, in Pytorch
parti-pytorch
Implementation of Parti, Google's pure attention-based text-to-image neural network, in Pytorch
magvit2-pytorch
Implementation of MagViT2 Tokenizer in Pytorch
recurrent-memory-transformer-pytorch
Implementation of Recurrent Memory Transformer, Neurips 2022 paper, in Pytorch
robotic-transformer-pytorch
Implementation of RT1 (Robotic Transformer) in Pytorch
iTransformer
Unofficial implementation of iTransformer - SOTA Time Series Forecasting using Attention networks, out of Tsinghua / Ant group
q-transformer
Implementation of Q-Transformer, Scalable Offline Reinforcement Learning via Autoregressive Q-Functions, out of Google Deepmind
classifier-free-guidance-pytorch
Implementation of Classifier Free Guidance in Pytorch, with emphasis on text conditioning, and flexibility to include multiple text embedding models
BS-RoFormer
Implementation of Band Split Roformer, SOTA Attention network for music source separation out of ByteDance AI Labs
equiformer-pytorch
Implementation of the Equiformer, SE3/E3 equivariant attention network that reaches new SOTA, and adopted for use by EquiFold for protein folding
En-transformer
Implementation of E(n)-Transformer, which incorporates attention mechanisms into Welling's E(n)-Equivariant Graph Neural Network
h-transformer-1d
Implementation of H-Transformer-1D, Hierarchical Attention for Sequence Learning
llama-qrlhf
Implementation of the Llama architecture with RLHF + Q-learning
nystrom-attention
Implementation of Nyström Self-attention, from the paper Nyströmformer
bidirectional-cross-attention
A simple cross attention that updates both the source and target in one step
pytorch-custom-utils
Just some miscellaneous utility functions / decorators / modules related to Pytorch and Accelerate to help speed up implementation of new AI research
mirasol-pytorch
Implementation of 🌻 Mirasol, SOTA Multimodal Autoregressive model out of Google Deepmind, in Pytorch
gateloop-transformer
Implementation of GateLoop Transformer in Pytorch and Jax
taylor-series-linear-attention
Explorations into the recently proposed Taylor Series Linear Attention
agent-attention-pytorch
Implementation of Agent Attention in Pytorch
gradnorm-pytorch
A practical implementation of GradNorm, Gradient Normalization for Adaptive Loss Balancing, in Pytorch
diffusion-policy
Implementation of Diffusion Policy, Toyota Research's supposed breakthrough in leveraging DDPMs for learning policies for real-world Robotics
AMIE-pytorch
Implementation of the general framework for AMIE, from the paper "Towards Conversational Diagnostic AI", out of Google Deepmind
evolutionary-design-molecules
Implementation of the algorithm detailed in paper "Evolutionary design of molecules based on deep learning and a genetic algorithm"