ModelTC

lightllm

LightLLM is a Python-based LLM (Large Language Model) inference and serving framework, notable for its lightweight design, easy scalability, and high-speed performance.

MQBench

Model Quantization Benchmark

United-Perception

United Perception

llmc

This is the official PyTorch implementation of "LLM-QBench: A Benchmark Towards the Best Practice for Post-training Quantization of Large Language Models", and also an efficient LLM compression tool with various advanced compression methods, supporting multiple inference backends.

Dipoorlet

Offline Quantization Tools for Deploy.

awesome-lm-system

Summary of system papers/frameworks/codes/tools on training or serving large model

TFMQ-DM

[CVPR 2024 Highlight] This is the official PyTorch implementation of "TFMQ-DM: Temporal Feature Maintenance Quantization for Diffusion Models".

NART

NART = NART is not A RunTime, a deep learning inference framework.

Outlier_Suppression_Plus

Official implementation of the EMNLP23 paper: Outlier Suppression+: Accurate quantization of large language models by equivalent and optimal shifting and scaling

NNLQP

EasyLLM

Built upon Megatron-Deepspeed and HuggingFace Trainer, EasyLLM has reorganized the code logic with a focus on usability. While enhancing usability, it also ensures training efficiency.

LPCV2021_Winner_Solution

QLLM

[ICLR 2024] This is the official PyTorch implementation of "QLLM: Accurate and Efficient Low-Bitwidth Quantization for Large Language Models"

pyvlova

Yet another Polyhedra Compiler for DeepLearning

LPCV_2023_solution

AAAI2023_EAMPD

AAAI2023 Efficient and Accurate Models towards Practical Deep Learning Baseline

Prototype

L2_Compression

Imagenet-S

Robustness for real-world system noise

msbench

A tool for model sparse based on torch.fx

ChatGLM-6B

ChatGLM-6B: An Open Bilingual Dialogue Language Model | 开源双语对话语言模型

general-sam

A general suffix automaton implementation in Rust with Python bindings

mtc-token-healing

Token healing implementation in Rust

FCPTS

general-sam-py

Python bindings for general-sam and some utilities

statecs

DeepSpeed

DeepSpeed is a deep learning optimization library that makes distributed training and inference easy, efficient, and effective.

greedy-tokenizer

Greedily tokenize strings with the longest tokens iteratively.

systemnoise_web

UP_LPCV2023_Plugin