KellerJordan

cifar10-airbench

94% on CIFAR-10 in 3.29 seconds 💨 https://arxiv.org/abs/2404.00498

REPAIR

Code release for REPAIR: REnormalizing Permuted Activations for Interpolation Repair

hlb-CIFAR10

Train to 94% on CIFAR-10 in 4.4 seconds on a single A100

top-sgd

Optimization algorithm which fits a ResNet to CIFAR-10 5x faster than SGD / Adam (with terrible generalization)

cifar10-loader

Fast and easy to use CIFAR-10 dataloader

Exponentiated-Gradient-PyTorch

EG plus/minus optimizer implemented in PyTorch

elastic-airbench

CIFAR10-isolated-rng

CIFAR-10 training script with separate seeds for model initialization, data ordering, and data augmentation

pixelated-features-bugs

Code for training and evaluating the robustness of models using pixelated data

C-http-server

A simple HTTP server in C, for educational purposes.

cifar10-zoo

development area for cifar10 tech

clean-cifar10-airbench

Variant of cifar10-airbench which removes several tricks. Ideal for research

Coding-Problems

Congress-Data

Explore congressional bill data

ffcv

FFCV: Fast Forward Computer Vision (and other ML workloads!)

ffcv-cifar

Train a large number of CIFAR-10 models using FFCV

ffcv-imagenet

Train ImageNet *fast* in 500 lines of code with FFCV -- forked for training only Resnet18s

flash-attention

Fast and memory-efficient exact attention

git-re-basin

Code release for "Git Re-Basin: Merging Models modulo Permutation Symmetries"

gpt-sandbox

karpathy.github.io

my blog

llm.c

LLM training in simple, raw C/CUDA

math-notebooks

mostly jupyter notebooks for probability theory tests

Megatron-LM

Ongoing research training transformer models at scale

nanoGPT

The simplest, fastest repository for training/finetuning medium-sized GPTs.

picoGPT

fork of nanoGPT

raw-webgl

robustness-featsnotbugs

A replication of "Adversarial Examples Are Not Bugs, They Are Features" https://arxiv.org/abs/1905.02175

sandboxes

trak

A fast, effective data attribution method for neural networks in PyTorch