quantization

There are 32 repositories under quantization topic.

LLaMA-Factory
hiyouga / LLaMA-Factory
Unify Efficient Fine-Tuning of 100+ LLMs
agent ai chatglm fine-tuning gpt instruction-tuning language-model large-language-models llama llama3 llm lora mistral moe peft qlora quantization qwen rlhf transformers
Language:Python 22133
Chinese-LLaMA-Alpaca
ymcui / Chinese-LLaMA-Alpaca
中文LLaMA&Alpaca大语言模型+本地CPU/GPU训练部署 (Chinese LLaMA & Alpaca LLMs)
llm plm pre-trained-language-models alpaca llama nlp quantization large-language-models lora alpaca-2 llama-2
Language:Python 17537
SYSTRAN / faster-whisper
Faster Whisper transcription with CTranslate2
deep-learning inference quantization speech-recognition speech-to-text transformer whisper openai
Language:Python 9176
Qbot
UFund-Me / Qbot
[🔥updating ...] AI 自动量化交易机器人 AI-powered Quantitative Investment Research Platform. 📃 online docs: https://ufund-me.github.io/Qbot ✨ :news: qbot-mini: https://github.com/Charmve/iQuant
bitcoin blockchain deep-learning fintech funds machine-learning pytrade qlib quant-trade quant-trader quantitative-finance quantitative-trading quantization strategies trade-bot trademarks
Language:Jupyter Notebook 5997
kornelski / pngquant
Lossy PNG compressor — pngquant command based on libimagequant library
c conversion image-optimization palette png png-compression pngquant quality quantization smaller stdin
Language:C 5036
IntelLabs / distiller
Neural Network Distiller by Intel AI Lab: a Python package for neural network compression research. https://intellabs.github.io/distiller
pytorch pruning quantization pruning-structures jupyter-notebook network-compression deep-neural-networks regularization group-lasso distillation early-exit truncated-svd onnx automl-for-compression
Language:Jupyter Notebook 4309
AutoGPTQ / AutoGPTQ
An easy-to-use LLMs quantization package with user-friendly apis, based on GPTQ algorithm.
deep-learning inference large-language-models llms nlp pytorch quantization transformer transformers
Language:Python 3859
huawei-noah / Pretrained-Language-Model
Pretrained language model and its related optimization techniques developed by Huawei Noah's Ark Lab.
knowledge-distillation model-compression quantization pretrained-models large-scale-distributed
Language:Python 2963
nlp-architect
IntelLabs / nlp-architect
A model library for exploring state-of-the-art deep learning topologies and techniques for optimizing Natural Language Processing neural networks
deeplearning nlp nlu tensorflow dynet deep-learning pytorch bert transformers quantization
Language:Python 2930
deepsparse
neuralmagic / deepsparse
Sparsity-aware deep learning inference runtime for CPUs
computer-vision cpus deepsparse inference llm-inference machinelearning nlp object-detection onnx performance pretrained-models pruning quantization sparsification
Language:Python 2881
OpenNMT / CTranslate2
Fast inference engine for Transformer models
neural-machine-translation cpp mkl quantization cuda thrust opennmt deep-neural-networks openmp onednn intrinsics avx2 avx parallel-computing gemm neon transformer-models machine-translation deep-learning inference
Language:C++ 2851
aaron-xichen / pytorch-playground
Base pretrained models and datasets in pytorch (MNIST, SVHN, CIFAR10, CIFAR100, STL10, AlexNet, VGG16, VGG19, ResNet, Inception, SqueezeNet)
pytorch-tutorial pytorch-tutorials pytorch quantization
Language:Python 2561
stochasticai / xTuring
Build, customize and control you own LLMs. From data pre-processing to fine-tuning, xTuring provides an easy way to personalize open-source LLMs. Join our discord community: https://discord.gg/TgHXuSJEk6
deep-learning fine-tuning gpt-2 gpt-j llama llm lora language-model alpaca finetuning adapter gen-ai generative-ai mistral mixed-precision peft quantization
Language:Python 2528
dvmazur / mixtral-offloading
Run Mixtral-8x7B models in Colab or consumer desktops
colab-notebook deep-learning google-colab language-model llm mixture-of-experts offloading pytorch quantization
Language:Python 2244
huggingface / optimum
🚀 Accelerate training and inference of 🤗 Transformers and 🤗 Diffusers with easy to use hardware optimization tools
onnx pytorch inference training intel graphcore onnxruntime transformers quantization habana optimization tflite
Language:Python 2190
666DZY666 / micronet
micronet, a model compression and deploy lib. compression: 1、quantization: quantization-aware-training(QAT), High-Bit(>2b)(DoReFa/Quantization and Training of Neural Networks for Efficient Integer-Arithmetic-Only Inference)、Low-Bit(≤2b)/Ternary and Binary(TWN/BNN/XNOR-Net); post-training-quantization(PTQ), 8-bit(tensorrt); 2、 pruning: normal、regular and group convolutional channel pruning; 3、 group convolution structure; 4、batch-normalization fuse for quantization. deploy: tensorrt, fp32/fp16/int8(ptq-calibration)、op-adapt(upsample)、dynamic_shape
quantization pruning dorefa twn bnn xnor-net pytorch model-compression group-convolution network-slimming neuromorphic-computing convolutional-networks network-in-network integer-arithmetic-only quantization-aware-training post-training-quantization tensorrt onnx tensorrt-int8-python batch-normalization-fuse
Language:Python 2183
intel / neural-compressor
SOTA low-bit LLM quantization (INT8/FP8/INT4/FP4/NF4) & sparsity; leading model compression techniques on TensorFlow, PyTorch, and ONNX Runtime
low-precision pruning sparsity auto-tuning knowledge-distillation quantization quantization-aware-training post-training-quantization smoothquant large-language-models awq fp4 gptq int4 int8 mxformat sparsegpt
Language:Python 1994
quic / aimet
AIMET is a library that provides advanced quantization and compression techniques for trained neural network models.
quantization deep-learning compression open-source machine-learning pruning auto-ml network-compression deep-neural-networks network-quantization opensource
Language:Python 1930
htqin / awesome-model-quantization
A list of papers, docs, codes about model quantization. This repo is aimed to provide the info for model quantization research, we are continuously improving the project. Welcome to PR the works (papers, repositories) that are missed by the repo.
deep-learning quantization awesome model-compression binarization binarized-neural-networks binary-network efficient-deep-learning lightweight-neural-network model-acceleration model-quantization
1647
PaddlePaddle / PaddleSlim
PaddleSlim is an open-source library for deep model compression and architecture search.
pruning quantization nas bert compression detection distillation ernie segmentation sparsity tensorrt transformer yolov6 yolov5 yolov7
Language:Python 1526
tensorflow / model-optimization
A toolkit to optimize ML models for deployment for Keras and TensorFlow, including quantization and pruning.
tensorflow machine-learning deep-learning optimization quantized-neural-networks quantized-networks quantized-training keras model-compression compression ml pruning sparsity quantization
Language:Python 1471
openppl-public / ppq
PPL Quantization Tool (PPQ) is a powerful offline neural network quantization tool.
caffe cuda deep-learning neural-network onnx open-source pytorch quantization
Language:Python 1381
intel / intel-extension-for-pytorch
A Python package for extending the official PyTorch that can easily obtain performance on Intel platform
pytorch neural-network machine-learning deep-learning intel quantization
Language:Python 1378
open-mmlab / mmrazor
OpenMMLab Model Compression Toolbox and Benchmark.
autoslim classification darts detection knowledge-distillation nas pruning pytorch quantization segmentation spos
Language:Python 1378
openvinotoolkit / training_extensions
Train, Evaluate, Optimize, Deploy Computer Vision Models via OpenVINO™
openvino computer-vision deep-learning pytorch neural-networks-compression quantization hyper-parameter-optimization image-classification image-segmentation object-detection self-supervised-learning semi-supervised-learning transfer-learning action-recognition anomaly-detection datumaro automl machine-learning incremental-learning
Language:Python 1121
huawei-noah / Efficient-Computing
Efficient computing methods developed by Huawei Noah's Ark Lab
knowledge-distillation model-compression binary-neural-networks pruning quantization self-supervised
Language:Jupyter Notebook 1118
RWKV / rwkv.cpp
INT4/INT5/INT8 and FP16 inference on CPU for RWKV language model
deep-learning ggml language-model llm machine-learning quantization rwkv
Language:C++ 1112
Xilinx / brevitas
Brevitas: neural network quantization in PyTorch
quantization pytorch brevitas fpga neural-networks hardware-acceleration xilinx deep-learning ptq qat
Language:Python 1097
openvinotoolkit / nncf
Neural Network Compression Framework for enhanced OpenVINO™ inference
quantization pruning sparsity quantization-aware-training mixed-precision-training compression semantic-segmentation object-detection classification nlp bert mmdetection transformers hawq pytorch tensorflow onnx openvino deep-learning
Language:Python 830
guan-yuan / awesome-AutoML-and-Lightweight-Models
A list of high-quality (newest) AutoML works and lightweight models including 1.) Neural Architecture Search, 2.) Lightweight Structures, 3.) Model Compression, Quantization and Acceleration, 4.) Hyperparameter Optimization, 5.) Automated Feature Engineering.
automl meta-learning automated-feature-engineering hyperparameter-optimization architecture-search model-compression model-acceleration awesome-list neural-architecture-search nas pytorch quantization quantized-neural-network quantized-training tensorflow
827
mit-han-lab / tinyengine
[NeurIPS 2020] MCUNet: Tiny Deep Learning on IoT Devices; [NeurIPS 2021] MCUNetV2: Memory-Efficient Patch-based Inference for Tiny Deep Learning; [NeurIPS 2022] MCUNetV3: On-Device Training Under 256KB Memory
c codegenerator cpp deep-learning microcontroller pytorch tinyml edge-computing neural-architecture-search quantization
Language:C 746
ImageOptim / libimagequant
Palette quantization library that powers pngquant and other PNG optimizers
quantization quality palette rgba-pixels palette-generation minification pixel-array image-pixels conversion visual-studio callback image-optimization pngquant
Language:Rust 730
csarron / awesome-emdl
Embedded and mobile deep learning research resources
efficient-neural-networks deep-neural-networks deep-learning mobile-ai embedded-ai mobile-inference mobile-deep-learning pruning quantization neural-network-compression inference
720
Xilinx / finn
Dataflow compiler for QNN inference on FPGAs
compiler dataflow fpga neural-network quantization
Language:Python 669
RahulSChand / gpu_poor
Calculate token/s & GPU memory requirement for any LLM. Supports llama.cpp/ggml/bnb/QLoRA quantization
ggml huggingface llm quantization gpu language-model pytorch llama llama2 llamacpp
Language:JavaScript 649
DeepVAC / deepvac
PyTorch Project Specification.
pytorch deepvac python tensorboard quantization amp torchscript onnx ncnn coreml ddp tensorrt
Language:Python 642