gyeongchan-yun

ml-systems-papers

Curated collection of papers in machine learning systems

Megatron-LLM

distributed trainer for LLMs

gpt-pytorch

PyTorch Implementation of OpenAI GPT

DT-FM

memory_profiler

Monitor Memory usage of Python code

VLDB2023_SDPipe

GPT2

PyTorch Implementation of OpenAI GPT-2

Merak

Oobleck

A resilient distributed training framework

EnvPipe

Megatron-DeepSpeed

Ongoing research training transformer language models at scale, including: BERT & GPT-2

pollux-results

Chimera

Chimera: Efficiently Training Large-Scale Neural Networks with Bidirectional Pipelines.

gavel

Code for "Heterogenity-Aware Cluster Scheduling Policies for Deep Learning Workloads", which appeared at OSDI 2020

KungFu

Fast and Adaptive Distributed Machine Learning for TensorFlow, PyTorch and MindSpore.

T2T-ViT

ICCV2021, Tokens-to-Token ViT: Training Vision Transformers from Scratch on ImageNet

pytorch-MNIST-CelebA-GAN-DCGAN

Pytorch implementation of Generative Adversarial Networks (GAN) and Deep Convolutional Generative Adversarial Networks (DCGAN) for MNIST and CelebA datasets

shockwave

Code for "Shockwave: Fair and Efficient Cluster Scheduling for Dynamic Adaptation in Machine Learning" [NSDI '23]

SimiGrad

Anonymous repo created for NIPS submission SimiGrad

Espresso

Hi-Speed DNN Training with Espresso: Unleashing the Full Potential of Gradient Compression with Near-Optimal Usage Strategies (EuroSys '23)

byteps

A high performance and generic framework for distributed DNN training

SimiGrad

Public Code for NIPS submission SimiGrad: Fine-Grained Adaptive Batching for Large ScaleTraining using Gradient Similarity Measurement

awesome-gnn-systems

A list of awesome GNN systems.

pyg_autoscale

Implementation of "GNNAutoScale: Scalable and Expressive Graph Neural Networks via Historical Embeddings" in PyTorch

marius

Large scale graph learning on a single machine.

DeepPlan

Fast and Efficient Model Serving Using Multi-GPUs with Direct-Host-Access (ACM EuroSys '23)

grace

GRACE - GRAdient ComprEssion for distributed deep learning

SHADE

SHADE: Enable Fundamental Cacheability for Distributed Deep Learning Training

adaptdl

Resource-adaptive cluster scheduler for deep learning training.

Awesome-ML-for-System

SOTA Learning-augmented Systems