C-TC

ThunderKittens

Tile primitives for speedy kernels

float8_experimental

This repository contains the experimental PyTorch native float8 training UX

AI-Chip

A list of ICs and IPs for AI, Machine Learning and Deep Learning.

scalene

Scalene: a high-performance, high-precision CPU, GPU, and memory profiler for Python with AI-powered optimization proposals

multi-gpu-programming-models

Examples demonstrating available options to program multiple GPUs in a single node or a cluster

torchtitan

A native PyTorch Library for large model training

mscclpp

MSCCL++: A GPU-driven communication stack for scalable AI applications

awesome-RLHF

A curated list of reinforcement learning with human feedback resources (continually updated)

Awesome_LLM_System-PaperList

Since the emergence of chatGPT in 2022, the acceleration of Large Language Model has become increasingly important. Here is a list of papers on accelerating LLMs, currently focusing mainly on inference acceleration, and related works will be gradually added in the future. Welcome contributions!

microxcaling

PyTorch emulation library for Microscaling (MX)-compatible data formats

attorch

A subset of PyTorch's neural network modules, written in Python using OpenAI's Triton.

lightllm

LightLLM is a Python-based LLM (Large Language Model) inference and serving framework, notable for its lightweight design, easy scalability, and high-speed performance.

msccl

Microsoft Collective Communication Library

kineto

A CPU+GPU Profiling library that provides access to timeline traces and hardware performance counters.

llamafia.github

TensorRT-LLM

TensorRT-LLM provides users with an easy-to-use Python API to define Large Language Models (LLMs) and build TensorRT engines that contain state-of-the-art optimizations to perform inference efficiently on NVIDIA GPUs. TensorRT-LLM also contains components to create Python and C++ runtimes that execute those TensorRT engines.

Pai-Megatron-Patch

The official repo of Pai-Megatron-Patch for LLM & VLM large scale training developed by Alibaba Cloud.

long-context-attention

Sequence Parallel Attention for Long Context LLM Model Training and Inference

LMFlow

An Extensible Toolkit for Finetuning and Inference of Large Foundation Models. Large Models for All.

ring-flash-attention

Ring attention implementation with flash attention

libai

LiBai(李白): A Toolbox for Large-Scale Distributed Parallel Training

yiyin

一款照片水印添加工具

Awesome-LLM-System-Papers

MS-AMP

Microsoft Automatic Mixed Precision Library

veScale

A PyTorch Native LLM Training Framework

ml-engineering

Machine Learning Engineering Open Book

nanotron

Minimalistic large language model 3D-parallelism training

superbenchmark

A validation and profiling tool for AI infrastructure

Megatron-LM

Ongoing research training transformer models at scale

xla

A machine learning compiler for GPUs, CPUs, and ML accelerators