CHEN Yuhan's starred repositories
llm_aided_ocr
Enhance Tesseract OCR output for scanned PDFs by applying Large Language Model (LLM) corrections.
TensorRT-Incubator
Experimental projects related to TensorRT
flash-attention-minimal
Flash Attention in ~100 lines of CUDA (forward pass only)
TensorRT-LLM
TensorRT-LLM provides users with an easy-to-use Python API to define Large Language Models (LLMs) and build TensorRT engines that contain state-of-the-art optimizations to perform inference efficiently on NVIDIA GPUs. TensorRT-LLM also contains components to create Python and C++ runtimes that execute those TensorRT engines.
MInference
To speed up Long-context LLMs' inference, approximate and dynamic sparse calculate the attention, which reduces inference latency by up to 10x for pre-filling on an A100 while maintaining accuracy.
sarathi-serve
A low-latency & high-throughput serving engine for LLMs
perf-ninja
This is an online course where you can learn and master the skill of low-level performance analysis and tuning.
awesome-local-ai
An awesome repository of local AI tools
LLaMA-Factory
A WebUI for Efficient Fine-Tuning of 100+ LLMs (ACL 2024)
cppinsights
C++ Insights - See your source code with the eyes of a compiler
Pruner-Zero
Pruner-Zero: Evolving Symbolic Pruning Metric from scratch for LLMs
compiler-and-arch
A list of tutorials, paper, talks, and open-source projects for emerging compiler and architecture
llama3-from-scratch
llama3 implementation one matrix multiplication at a time