dblalock

DeepSpeed

DeepSpeed is a deep learning optimization library that makes distributed training and inference easy, efficient, and effective.

applied-ml

📚 Papers & tech blogs by companies sharing their work on data science & machine learning in production.

modern-cpp-features

A cheatsheet of modern C++ language and library features.

hammerspoon

Staggeringly powerful macOS desktop automation with Lua

scalene

Scalene: a high-performance, high-precision CPU, GPU, and memory profiler for Python with AI-powered optimization proposals

ml-engineering

Machine Learning Engineering Open Book

pretrained-models.pytorch

Pretrained ConvNets for pytorch: NASNet, ResNeXt, ResNet, InceptionV4, InceptionResnetV2, Xception, DPN, etc.

einops

Flexible and powerful tensor operations for readable and reliable code (for pytorch, jax, TF and others)

composer

Supercharge Your Model Training

DALI

A GPU-accelerated library containing highly optimized building blocks and an execution engine for data processing to accelerate deep learning training and inference applications.

readyset

Readyset is a MySQL and Postgres wire-compatible caching layer that sits in front of existing databases to speed up queries and horizontally scale read throughput. Under the hood, ReadySet caches the results of cached select statements and incrementally updates these results over time as the underlying data changes.

llm-foundry

LLM training code for Databricks foundation models

annotated_latex_equations

Examples of how to create colorful, annotated equations in Latex using Tikz.

differential-privacy

Google's differential privacy libraries.

hlb-CIFAR10

Train to 94% on CIFAR-10 in <6.3 seconds on a single A100. Or ~95.79% in ~110 seconds (or less!)

Neighborhood-Attention-Transformer

Neighborhood Attention Transformer, arxiv 2022 / CVPR 2023. Dilated Neighborhood Attention Transformer, arxiv 2022

streaming

A Data Streaming Library for Efficient Neural Network Training

imagenette

A smaller subset of 10 easily classified classes from Imagenet, and a little more French

bagua

Bagua Speeds up PyTorch

ImageNet21K

Official Pytorch Implementation of: "ImageNet-21K Pretraining for the Masses"(NeurIPS, 2021) paper

uarch-bench

A benchmark for low-level CPU micro-architectural features

UniverSeg

UniverSeg: Universal Medical Image Segmentation

examples

Fast and flexible reference benchmarks

shrinkbench

PyTorch library to facilitate development and standardized evaluation of neural network pruning methods.

papers-with-video

A browser extension that links video explanations to research papers on arxiv.org

SGEMM_CUDA

Fast CUDA matrix multiplication from scratch

halutmatmul

Hashed Lookup Table based Matrix Multiplication (halutmatmul) - Stella Nera accelerator

custom_matmul_kernels

Customized matrix multiplication kernels

hfta

Boost hardware utilization for ML training workloads via Inter-model Horizontal Fusion

composer-hydra-template