Jiahao Tan's starred repositories
Megatron-LM
Ongoing research training transformer models at scale
cuda-samples
Samples for CUDA Developers which demonstrates features in CUDA Toolkit
DeepSeek-V2
DeepSeek-V2: A Strong, Economical, and Efficient Mixture-of-Experts Language Model
Awesome-LLM-Inference
📖A curated list of Awesome LLM Inference Paper with codes, TensorRT-LLM, vLLM, streaming-llm, AWQ, SmoothQuant, WINT8/4, Continuous Batching, FlashAttention, PagedAttention etc.
CUDA-Programming
Sample codes for my CUDA programming book
torchtitan
A native PyTorch Library for large model training
CUDALibrarySamples
CUDA Library Samples
how-to-optim-algorithm-in-cuda
how to optimize some algorithm in cuda.
flash-linear-attention
Efficient implementations of state-of-the-art linear attention models in Pytorch and Triton
CUDA-Learn-Notes
🎉CUDA/C++ 笔记 / 大模型手撕CUDA / 技术博客,更新随缘: flash_attn、sgemm、sgemv、warp reduce、block reduce、dot product、elementwise、softmax、layernorm、rmsnorm、hist etc.
CUDA-Optimization-Guide
Xiao's CUDA Optimization Guide [Active Adding New Contents]