Phil Wang's repositories
PaLM-rlhf-pytorch
Implementation of RLHF (Reinforcement Learning with Human Feedback) on top of the PaLM architecture. Basically ChatGPT but with PaLM
toolformer-pytorch
Implementation of Toolformer, Language Models That Can Use Tools, by MetaAI
gigagan-pytorch
Implementation of GigaGAN, new SOTA GAN out of Adobe. Culmination of nearly a decade of research into GANs
CoCa-pytorch
Implementation of CoCa, Contrastive Captioners are Image-Text Foundation Models, in Pytorch
RETRO-pytorch
Implementation of RETRO, Deepmind's Retrieval based Attention net, in Pytorch
tab-transformer-pytorch
Implementation of TabTransformer, attention network for tabular data, in Pytorch
parti-pytorch
Implementation of Parti, Google's pure attention-based text-to-image neural network, in Pytorch
egnn-pytorch
Implementation of E(n)-Equivariant Graph Neural Networks, in Pytorch
robotic-transformer-pytorch
Implementation of RT1 (Robotic Transformer) in Pytorch
spear-tts-pytorch
Implementation of Spear-TTS - multi-speaker text-to-speech attention network, in Pytorch
equiformer-pytorch
Implementation of the Equiformer, SE3/E3 equivariant attention network that reaches new SOTA, and adopted for use by EquiFold for protein folding
En-transformer
Implementation of E(n)-Transformer, which incorporates attention mechanisms into Welling's E(n)-Equivariant Graph Neural Network
simple-hierarchical-transformer
Experiments around a simple idea for inducing multiple hierarchical predictive model within a GPT
med-seg-diff-pytorch
Implementation of MedSegDiff in Pytorch - SOTA medical segmentation using DDPM and filtering of features in fourier space
metnet3-pytorch
Implementation of MetNet-3, SOTA neural weather model out of Google Deepmind, in Pytorch
llama-qrlhf
Implementation of the Llama architecture with RLHF + Q-learning
nystrom-attention
Implementation of Nyström Self-attention, from the paper Nyströmformer
medical-ai-experiments
A repository to house some personal attempts to beat some state-of-the-art for medical datasets
zorro-pytorch
Implementation of Zorro, Masked Multimodal Transformer, in Pytorch
mirasol-pytorch
Implementation of 🌻 Mirasol, SOTA Multimodal Autoregressive model out of Google Deepmind, in Pytorch
dreamcraft3d-pytorch
Implementation of Dreamcraft3D, 3D content generation in Pytorch
agent-attention-pytorch
Implementation of Agent Attention in Pytorch
diffusion-policy
Implementation of Diffusion Policy, Toyota Research's supposed breakthrough in leveraging DDPMs for learning policies for real-world Robotics
kalman-filtering-attention
Implementation of the Kalman Filtering Attention proposed in "Kalman Filtering Attention for User Behavior Modeling in CTR Prediction"
neural-plexer-pytorch
Implementation of Nvidia's NeuralPlexer, for end-to-end differentiable design of functional small-molecules and ligand-binding proteins, in Pytorch
pause-transformer
Yet another random morning idea to be quickly tried and architecture shared if it works; to allow the transformer to pause for any amount of time on any token
evolutionary-design-molecules
Implementation of the algorithm detailed in paper "Evolutionary design of molecules based on deep learning and a genetic algorithm"
holodeck-pytorch
Implementation of a holodeck, written in Pytorch