ZZK's repositories
cutlass_master
CUDA Templates for Linear Algebra Subroutines
APPy
APPy (Annotated Parallelism for Python) enables users to annotate loops and tensor expressions in Python with compiler directives akin to OpenMP, and automatically compiles the annotated code to GPU kernels.
attorch
A subset of PyTorch's neural network modules, written in Python using OpenAI's Triton.
auto-round
SOTA Weight-only Quantization Algorithm for LLMs
BitBLAS
BitBLAS is a library to support mixed-precision matrix multiplications, especially for quantized LLM deployment.
cccl
CUDA C++ Core Libraries
cudnn-frontend
cudnn_frontend provides a c++ wrapper for the cudnn backend API and samples on how to use it
EETQ
Easy and Efficient Quantization for Transformers
faster-nougat
Implementation of nougat that focuses on processing pdf locally.
fp6_llm
An efficient GPU support for LLM inference with 6-bit quantization (FP6).
GPUSorting
OneSweep, implemented in CUDA, D3D12, and Unity style compute shaders. Theoretically portable to all wave/warp/subgroup sizes.
KIVI
KIVI: A Tuning-Free Asymmetric 2bit Quantization for KV Cache
KVQuant
KVQuant: Towards 10 Million Context Length LLM Inference with KV Cache Quantization
lightning-thunder
Make PyTorch models up to 40% faster! Thunder is a source to source compiler for PyTorch. It enables using different hardware executors at once; across one or thousands of GPUs.
LLMRoofline
Compare different hardware platforms via the Roofline Model for LLM inference tasks.
open-gpu-kernel-modules
NVIDIA Linux open GPU with P2P support
qllm-eval
Code Repository of Evaluating Quantized Large Language Models
quanto
A pytorch Quantization Toolkit
TensorRT-Model-Optimizer
TensorRT Model Optimizer is a unified library of state-of-the-art model optimization techniques such as quantization and sparsity. It compresses deep learning models for downstream deployment frameworks like TensorRT-LLM or TensorRT to optimize inference speed on NVIDIA GPUs.
ThunderKittens
Tile primitives for speedy kernels
tiny-gpu
A minimal GPU design in Verilog to learn how GPUs work from the ground up
triton
Development repository for the Triton language and compiler
Triton-Puzzles
Puzzles for learning Triton