ThanThoai / TorchSlim

A Pytorch Optimization library for benchmarking and extending works in the domains of Knowledge Distillation, Pruning, and Quantization.

Supported Features

Pruning (Not support Conv3d)

• AGP Pruner (L-norm, APoZ)
• Slim Pruner (ChannlePruner, FilterPruner, GroupPruner, ...)
• FPGM Pruner
• ADMM Pruner
• Sensitivity Prunner

Quantization

• Support PQT, QAT, DorefaQuantization, v.v

Knowledge Distillation

Easy traning with student, teacher.

• Add example apply KD for training pruned model.
• Add reviewKD Distilling Knowledge via Knowledge Review

How to use:

pip install -r requirments.txt

• Document: Comming soon!!!!!

• Get infomation: Link

Roadmap

• Add pruning method for Transformer, RNN.
• Add quantization method.
• Add more loss function in Knowledge Distillation method.
• Add Spatial SVD method: Tensor decomposition technique to split a large layer in two smaller ones.
• Add visualization in tensorboard.