henryhmko

DeepSpeed

DeepSpeed is a deep learning optimization library that makes distributed training and inference easy, efficient, and effective.

LLMs-from-scratch

Implement a ChatGPT-like LLM in PyTorch from scratch, step by step

tuning_playbook

A playbook for systematically maximizing the performance of deep learning models.

swarm

Educational framework exploring ergonomic, lightweight multi-agent orchestration. Managed by OpenAI Solution team.

triton

Development repository for the Triton language and compiler

GPU-Puzzles

Solve puzzles. Learn CUDA.

TensorRT-LLM

TensorRT-LLM provides users with an easy-to-use Python API to define Large Language Models (LLMs) and build TensorRT engines that contain state-of-the-art optimizations to perform inference efficiently on NVIDIA GPUs. TensorRT-LLM also contains components to create Python and C++ runtimes that execute those TensorRT engines.

sglang

SGLang is a fast serving framework for large language models and vision language models.

Liger-Kernel

Efficient Triton Kernels for LLM Training

equinox

Elegant easy-to-use neural networks + scientific computing in JAX. https://docs.kidger.site/equinox/

TransformerEngine

A library for accelerating Transformer models on NVIDIA GPUs, including using 8-bit floating point (FP8) precision on Hopper and Ada GPUs, to provide better performance with lower memory utilization in both training and inference.

ThunderKittens

Tile primitives for speedy kernels

ao

PyTorch native quantization and sparsity for training and inference

awesome-jax

JAX - A curated list of resources https://github.com/google/jax

ArcticDB

ArcticDB is a high performance, serverless DataFrame database built for the Python Data Science ecosystem.

esm

lightning-thunder

Make PyTorch models up to 40% faster! Thunder is a source to source compiler for PyTorch. It enables using different hardware executors at once; across one or thousands of GPUs.

Triton-Puzzles

Puzzles for learning Triton

awesome-mixture-of-experts

A collection of AWESOME things about mixture-of-experts

melange-nvim

🗡️ Warm color scheme for Neovim and beyond

MS-AMP

Microsoft Automatic Mixed Precision Library

snowflake-arctic

Awesome-GPU

Awesome resources for GPUs

MS-SNSD

The Microsoft Scalable Noisy Speech Dataset (MS-SNSD) is a noisy speech dataset that can scale to arbitrary sizes depending on the number of speakers, noise types, and Speech to Noise Ratio (SNR) levels desired.

hardware-effects-gpu

Demonstration of various hardware effects on CUDA GPUs.

gpu-benches

collection of benchmarks to measure basic GPU capabilities

lolcats

Repo for "LoLCATs: On Low-Rank Linearizing of Large Language Models"

Awesome-Triton-Kernels

Collection of kernels written in Triton language

gpt-jax

Jax/Flax rewrite of Karpathy's nanoGPT

cs265

Website for CS 265