Wei Huang's starred repositories
arena-hard-auto
Arena-Hard-Auto: An automatic LLM benchmark.
TransformerCompression
For releasing code related to compression methods for transformers, accompanying our publications
FlagAttention
A collection of memory efficient attention operators implemented in the Triton language.
RAG-Retrieval
Unify Efficient Fine-tuning of RAG Retrieval, including Embedding, ColBERT,Cross Encoder
FasterTransformer
Transformer related optimization, including BERT, GPT
opencompass
OpenCompass is an LLM evaluation platform, supporting a wide range of models (Llama3, Mistral, InternLM2,GPT-4,LLaMa2, Qwen,GLM, Claude, etc) over 100+ datasets.
Liger-Kernel
Efficient Triton Kernels for LLM Training
TensorRT-LLM
TensorRT-LLM provides users with an easy-to-use Python API to define Large Language Models (LLMs) and build TensorRT engines that contain state-of-the-art optimizations to perform inference efficiently on NVIDIA GPUs. TensorRT-LLM also contains components to create Python and C++ runtimes that execute those TensorRT engines.
flashinfer
FlashInfer: Kernel Library for LLM Serving
any-precision-llm
[ICML 2024 Oral] Any-Precision LLM: Low-Cost Deployment of Multiple, Different-Sized LLMs
TinyChatEngine
TinyChatEngine: On-Device LLM Inference Library
smoothquant
[ICML 2023] SmoothQuant: Accurate and Efficient Post-Training Quantization for Large Language Models
GPTQ-triton
GPTQ inference Triton kernel