xiaobochen123

vllm

A high-throughput and memory-efficient inference and serving engine for LLMs

sglang

SGLang is yet another fast serving framework for large language models and vision language models.

AutoAWQ

AutoAWQ implements the AWQ algorithm for 4-bit quantization with a 2x speedup during inference. Documentation:

onnx

Open standard for machine learning interoperability

SampleCode

cutlass

CUDA Templates for Linear Algebra Subroutines

how-to-optimize-gemm

YHs_Sample

Yinghan's Code Sample

CppTemplateTutorial

中文的C++ Template的教学指南。与知名书籍C++ Templates不同，该系列教程将C++ Templates作为一门图灵完备的语言来讲授，以求帮助读者对Meta-Programming融会贯通。(正在施工中)

optimizer

Actively maintained ONNX Optimizer