quantization

There are 47 repositories under quantization topic.

LLaMA-Factory
hiyouga / LLaMA-Factory
Unified Efficient Fine-Tuning of 100+ LLMs & VLMs (ACL 2024)
agent ai deepseek fine-tuning gemma gpt instruction-tuning large-language-models llama llama3 llm lora moe nlp peft qlora quantization qwen rlhf transformers
Language:Python 62147
Chinese-LLaMA-Alpaca
ymcui / Chinese-LLaMA-Alpaca
中文LLaMA&Alpaca大语言模型+本地CPU/GPU训练部署 (Chinese LLaMA & Alpaca LLMs)
alpaca alpaca-2 large-language-models llama llama-2 llm lora nlp plm pre-trained-language-models quantization
Language:Python 18944
SYSTRAN / faster-whisper
Faster Whisper transcription with CTranslate2
deep-learning inference openai quantization speech-recognition speech-to-text transformer whisper
Language:Python 18936
Qbot
UFund-Me / Qbot
[🔥updating ...] AI 自动量化交易机器人(完全本地部署) AI-powered Quantitative Investment Research Platform. 📃 online docs: https://ufund-me.github.io/Qbot ✨ :news: qbot-mini: https://github.com/Charmve/iQuant
backtest bitcoin blockchain deep-learning fintech funds machine-learning pytrade qlib quant-trade quant-trader quantitative-finance quantitative-trading quantization strategies trade-bot trademarks
Language:Jupyter Notebook 14961
bitsandbytes
bitsandbytes-foundation / bitsandbytes
Accessible large language models via k-bit quantization for PyTorch.
llm machine-learning pytorch qlora quantization
Language:Python 7736
kornelski / pngquant
Lossy PNG compressor — pngquant command based on libimagequant library
pngquant quality png png-compression quantization stdin palette smaller conversion image-optimization c
Language:C 5467
AutoGPTQ / AutoGPTQ
An easy-to-use LLMs quantization package with user-friendly apis, based on GPTQ algorithm.
transformers deep-learning inference large-language-models llms nlp pytorch quantization transformer
Language:Python 4987
IntelLabs / distiller
Neural Network Distiller by Intel AI Lab: a Python package for neural network compression research. https://intellabs.github.io/distiller
automl-for-compression deep-neural-networks distillation early-exit group-lasso jupyter-notebook network-compression onnx pruning pruning-structures pytorch quantization regularization truncated-svd
Language:Jupyter Notebook 4402
OpenNMT / CTranslate2
Fast inference engine for Transformer models
neural-machine-translation cpp mkl quantization cuda thrust opennmt deep-neural-networks openmp onednn intrinsics avx2 avx parallel-computing gemm neon transformer-models machine-translation deep-learning inference
Language:C++ 4127
nunchaku-tech / nunchaku
[ICLR2025 Spotlight] SVDQuant: Absorbing Outliers by Low-Rank Components for 4-Bit Diffusion Models
diffusion-models flux genai lora mlsys quantization iclr iclr2025 comfyui
Language:Python 3312
huggingface / optimum
🚀 Accelerate inference and training of 🤗 Transformers, Diffusers, TIMM and Sentence Transformers with easy to use hardware optimization tools
graphcore habana inference intel onnx onnxruntime optimization pytorch quantization tflite training transformers
Language:Python 3161
deepsparse
neuralmagic / deepsparse
Sparsity-aware deep learning inference runtime for CPUs
computer-vision cpus deepsparse inference llm-inference machinelearning nlp object-detection onnx performance pretrained-models pruning quantization sparsification
Language:Python 3160
huawei-noah / Pretrained-Language-Model
Pretrained language model and its related optimization techniques developed by Huawei Noah's Ark Lab.
knowledge-distillation model-compression quantization pretrained-models large-scale-distributed
Language:Python 3143
nlp-architect
IntelLabs / nlp-architect
A model library for exploring state-of-the-art deep learning topologies and techniques for optimizing Natural Language Processing neural networks
bert deep-learning deeplearning dynet nlp nlu pytorch quantization tensorflow transformers
Language:Python 2938
aaron-xichen / pytorch-playground
Base pretrained models and datasets in pytorch (MNIST, SVHN, CIFAR10, CIFAR100, STL10, AlexNet, VGG16, VGG19, ResNet, Inception, SqueezeNet)
pytorch-tutorial pytorch-tutorials pytorch quantization
Language:Python 2702
stochasticai / xTuring
Build, personalize and control your own LLMs. From data pre-processing to fine-tuning, xTuring provides an easy way to personalize open-source LLMs. Join our discord community: https://discord.gg/TgHXuSJEk6
adapter deep-learning fine-tuning finetuning gen-ai generative-ai gpt-2 gpt-j language-model llama llm lora mistral mixed-precision peft quantization
Language:Python 2662
thu-ml / SageAttention
[ICLR2025, ICML2025, NeurIPS2025 Spotlight] Quantized Attention achieves speedup of 2-5x compared to FlashAttention, without losing end-to-end metrics across language, image, and video models.
attention inference-acceleration llm quantization cuda triton video-generation efficient-attention mlsys llm-infra vit video-generate
Language:Cuda 2640
intel / neural-compressor
SOTA low-bit LLM quantization (INT8/FP8/MXFP8/INT4/MXFP4/NVFP4) & sparsity; leading model compression techniques on TensorFlow, PyTorch, and ONNX Runtime
low-precision pruning sparsity auto-tuning knowledge-distillation quantization quantization-aware-training post-training-quantization smoothquant large-language-models awq fp4 gptq int4 int8 mxformat sparsegpt
Language:Python 2520
pytorch / ao
PyTorch native quantization and sparsity for training and inference
brrr dtypes inference mx pytorch quantization sparsity training float8 transformer offloading optimizer cuda llama
Language:Python 2495
quic / aimet
AIMET is a library that provides advanced quantization and compression techniques for trained neural network models.
auto-ml compression deep-learning deep-neural-networks machine-learning network-compression network-quantization open-source opensource pruning quantization
Language:Python 2492
nunchaku-tech / ComfyUI-nunchaku
ComfyUI Plugin of Nunchaku
comfyui diffusion genai mlsys quantization flux
Language:Python 2468
dvmazur / mixtral-offloading
Run Mixtral-8x7B models in Colab or consumer desktops
colab-notebook deep-learning google-colab language-model llm mixture-of-experts offloading pytorch quantization
Language:Python 2325
666DZY666 / micronet
micronet, a model compression and deploy lib. compression: 1、quantization: quantization-aware-training(QAT), High-Bit(>2b)(DoReFa/Quantization and Training of Neural Networks for Efficient Integer-Arithmetic-Only Inference)、Low-Bit(≤2b)/Ternary and Binary(TWN/BNN/XNOR-Net); post-training-quantization(PTQ), 8-bit(tensorrt); 2、 pruning: normal、regular and group convolutional channel pruning; 3、 group convolution structure; 4、batch-normalization fuse for quantization. deploy: tensorrt, fp32/fp16/int8(ptq-calibration)、op-adapt(upsample)、dynamic_shape
quantization pruning dorefa twn bnn xnor-net pytorch model-compression group-convolution network-slimming neuromorphic-computing convolutional-networks network-in-network integer-arithmetic-only quantization-aware-training post-training-quantization tensorrt onnx tensorrt-int8-python batch-normalization-fuse
Language:Python 2263
Efficient-ML / Awesome-Model-Quantization
A list of papers, docs, codes about model quantization. This repo is aimed to provide the info for model quantization research, we are continuously improving the project. Welcome to PR the works (papers, repositories) that are missed by the repo.
awesome binarized-neural-networks binary-network deep-learning efficient-deep-learning lightweight-neural-network model-acceleration model-compression model-quantization quantization
2263
vllm-project / llm-compressor
Transformers-compatible library for applying various compression algorithms to LLMs for optimized deployment with vLLM
compression quantization sparsity
Language:Python 2214
intel / intel-extension-for-pytorch
A Python package for extending the official PyTorch that can easily obtain performance on Intel platform
deep-learning intel machine-learning neural-network pytorch quantization
Language:Python 1988
OpenPPL / ppq
PPL Quantization Tool (PPQ) is a powerful offline neural network quantization tool.
neural-network deep-learning quantization pytorch caffe onnx cuda open-source
Language:Python 1766
open-mmlab / mmrazor
OpenMMLab Model Compression Toolbox and Benchmark.
nas pruning knowledge-distillation spos darts autoslim detection segmentation classification pytorch quantization
Language:Python 1640
PaddlePaddle / PaddleSlim
PaddleSlim is an open-source library for deep model compression and architecture search.
pruning quantization nas bert compression detection distillation ernie segmentation sparsity tensorrt transformer yolov6 yolov5 yolov7
Language:Python 1610
tensorflow / model-optimization
A toolkit to optimize ML models for deployment for Keras and TensorFlow, including quantization and pruning.
tensorflow machine-learning deep-learning optimization quantized-neural-networks quantized-networks quantized-training keras model-compression compression ml pruning sparsity quantization
Language:Python 1558
RWKV / rwkv.cpp
INT4/INT5/INT8 and FP16 inference on CPU for RWKV language model
deep-learning language-model llm machine-learning quantization rwkv ggml
Language:C++ 1552
adithya-s-k / AI-Engineering.academy
Mastering Applied AI, One Concept at a Time
fine-tuning finetuning finetuning-llms inference large-language-models llm python quantization
Language:Jupyter Notebook 1529
Xilinx / brevitas
Brevitas: neural network quantization in PyTorch
quantization pytorch brevitas fpga neural-networks hardware-acceleration xilinx deep-learning ptq qat
Language:Python 1424
RahulSChand / gpu_poor
Calculate token/s & GPU memory requirement for any LLM. Supports llama.cpp/ggml/bnb/QLoRA quantization
ggml huggingface llm quantization gpu language-model pytorch llama llama2 llamacpp
Language:JavaScript 1379
huawei-noah / Efficient-Computing
Efficient computing methods developed by Huawei Noah's Ark Lab
binary-neural-networks knowledge-distillation model-compression pruning quantization self-supervised
Language:Jupyter Notebook 1300
open-edge-platform / training_extensions
Train, Evaluate, Optimize, Deploy Computer Vision Models via OpenVINO™
openvino computer-vision deep-learning pytorch neural-networks-compression quantization hyper-parameter-optimization image-classification image-segmentation object-detection self-supervised-learning semi-supervised-learning transfer-learning action-recognition anomaly-detection datumaro automl machine-learning incremental-learning geti
Language:Python 1203

quantization

hiyouga / LLaMA-Factory

ymcui / Chinese-LLaMA-Alpaca

SYSTRAN / faster-whisper

UFund-Me / Qbot

bitsandbytes-foundation / bitsandbytes

kornelski / pngquant

AutoGPTQ / AutoGPTQ

IntelLabs / distiller

OpenNMT / CTranslate2

nunchaku-tech / nunchaku

huggingface / optimum

neuralmagic / deepsparse

huawei-noah / Pretrained-Language-Model

IntelLabs / nlp-architect

aaron-xichen / pytorch-playground

stochasticai / xTuring

thu-ml / SageAttention

intel / neural-compressor

pytorch / ao

quic / aimet

nunchaku-tech / ComfyUI-nunchaku

dvmazur / mixtral-offloading

666DZY666 / micronet

Efficient-ML / Awesome-Model-Quantization

vllm-project / llm-compressor

intel / intel-extension-for-pytorch

OpenPPL / ppq

open-mmlab / mmrazor

PaddlePaddle / PaddleSlim

tensorflow / model-optimization

RWKV / rwkv.cpp

adithya-s-k / AI-Engineering.academy

Xilinx / brevitas

RahulSChand / gpu_poor

huawei-noah / Efficient-Computing

open-edge-platform / training_extensions