sheng-qin

GACT-ICML

materials_discovery

graphcast

Open-Sora

Open-Sora: Democratizing Efficient Video Production for All

latent-diffusion

High-Resolution Image Synthesis with Latent Diffusion Models

DiT

Official PyTorch Implementation of "Scalable Diffusion Models with Transformers"

edm

Elucidating the Design Space of Diffusion-Based Generative Models (EDM)

msccl

Microsoft Collective Communication Library

DeepSeek-V2

DeepSeek-V2: A Strong, Economical, and Efficient Mixture-of-Experts Language Model

awesome-rdma

A curated list of awesome rdma resources

MatmulTutorial

A Easy-to-understand TensorOp Matmul Tutorial

h100-performance-tests

Awesome-LLM-System-Papers

LLM-FP4

The official implementation of the EMNLP 2023 paper LLM-FP4

apex

A PyTorch Extension: Tools for easy mixed precision and distributed training in Pytorch

Tensile

Stretching GPU performance for GEMMs and tensor contractions.

out-of-the-box-fp8-training

Demo of the unit_scaling library, showing how a model can be easily adapted to train in FP8.

veScale

A PyTorch Native LLM Training Framework

Model-References

Reference models for Intel(R) Gaudi(R) AI Accelerator

triton

Development repository for the Triton language and compiler

llama3

The official Meta Llama 3 GitHub site

MS-AMP

Microsoft Automatic Mixed Precision Library

NVTX

The NVIDIA® Tools Extension SDK (NVTX) is a C-based Application Programming Interface (API) for annotating events, code ranges, and resources in your applications.

microxcaling

PyTorch emulation library for Microscaling (MX)-compatible data formats

TransformerEngine

A library for accelerating Transformer models on NVIDIA GPUs, including using 8-bit floating point (FP8) precision on Hopper and Ada GPUs, to provide better performance with lower memory utilization in both training and inference.

how-to-optim-algorithm-in-cuda

how to optimize some algorithm in cuda.

mini-rv32ima

A tiny C header-only risc-v emulator.

cpufp

A CPU tool for benchmarking the peak of floating points

FEMFX

WebGL-Fluid-Simulation

Play with fluids in your browser (works even on mobile)