ModelTC

lightllm

LightLLM is a Python-based LLM (Large Language Model) inference and serving framework, notable for its lightweight design, easy scalability, and high-speed performance.

llmc

This is the official PyTorch implementation of "LLMC: Benchmarking Large Language Model Quantization with a Versatile Compression Toolkit".

OmniBal

TFMQ-DM

[CVPR 2024 Highlight] This is the official PyTorch implementation of "TFMQ-DM: Temporal Feature Maintenance Quantization for Diffusion Models".

EasyLLM

Built upon Megatron-Deepspeed and HuggingFace Trainer, EasyLLM has reorganized the code logic with a focus on usability. While enhancing usability, it also ensures training efficiency.

general-sam-py

Python bindings for general-sam and some utilities

mtc-token-healing

Token healing implementation in Rust

L2_Compression

msbench

A tool for model sparse based on torch.fx

MQBench

Model Quantization Benchmark

FCPTS

statecs

general-sam

A general suffix automaton implementation in Rust with Python bindings

DeepSpeed

DeepSpeed is a deep learning optimization library that makes distributed training and inference easy, efficient, and effective.

greedy-tokenizer

Greedily tokenize strings with the longest tokens iteratively.

QLLM

[ICLR 2024] This is the official PyTorch implementation of "QLLM: Accurate and Efficient Low-Bitwidth Quantization for Large Language Models"

Dipoorlet

Offline Quantization Tools for Deploy.

awesome-lm-system

Summary of system papers/frameworks/codes/tools on training or serving large model

LPCV_2023_solution

Outlier_Suppression_Plus

Official implementation of the EMNLP23 paper: Outlier Suppression+: Accurate quantization of large language models by equivalent and optimal shifting and scaling

UP_LPCV2023_Plugin

ChatGLM-6B

ChatGLM-6B: An Open Bilingual Dialogue Language Model | 开源双语对话语言模型

pyvlova

Yet another Polyhedra Compiler for DeepLearning

systemnoise_web

NART

NART = NART is not A RunTime, a deep learning inference framework.

United-Perception

United Perception

AAAI2023_EAMPD

AAAI2023 Efficient and Accurate Models towards Practical Deep Learning Baseline

NNLQP

Imagenet-S

Robustness for real-world system noise

LPCV2021_Winner_Solution