https://lightly.teamcode.com/dashboard
Pytorch Quantization
PyTorch-Quantization is a toolkit for training and evaluating PyTorch models with simulated quantization. Quantization can be added to the model automatically, or manually, allowing the model to be tuned for accuracy and performance. Quantization is compatible with NVIDIAs high performance integer kernels which leverage integer Tensor Cores. The quantized model can be exported to ONNX and imported by TensorRT 8.0 and later.
Install
Binaries
pip install pytorch-quantization --extra-index-url https://pypi.ngc.nvidia.comFrom Source
git clone https://github.com/NVIDIA/TensorRT.git
cd tools/pytorch-quantizationInstall PyTorch and prerequisites
pip install -r requirements.txt
# for CUDA 10.2 users
pip install torch>=1.9.1
# for CUDA 11.1 users
pip install torch>=1.9.1+cu111Build and install pytorch-quantization
# Python version >= 3.7, GCC version >= 5.4 required
python setup.py installNGC Container
pytorch-quantization is preinstalled in NVIDIA NGC PyTorch container, e.g. nvcr.io/nvidia/pytorch:22.12-py3
Resources
- Pytorch Quantization Toolkit userguide
- Quantization Basics whitepaper