da2so / DA2Lite

DA2Lite is an automated model compression toolkit for PyTorch.

DA2Lite (Deep Architecture to Lite) is a toolkit to compress and accelerate deep network models.

⭐ Star us on GitHub — it helps!!

	Frameworks & Libraries	Algorithms
Built-in	Supported Framework PyTorch Examples resnet18_tucker	Pruning L1 norm pruning L2 norm pruning EagleEye Slimming Filter Decomposition CP decomposition Tucker decomposition VBMF for rank selection Knowledge Distillation FSKD

Install

git clone https://github.com/da2so/DA2Lite.git

You will need a machine with a GPU and CUDA installed.
Then, you prepare runtime environment:

pip install -r requirements.txt

Use

Run

main.py(DA2Lite) runs with two main configurations like as follows:

CUDA_VISIBLE_DEVICES=0 python main.py --train_config_file=./configs/train/cifar10/cifar10/vgg16.yaml --compress_config_file=./configs/compress/tucker.yaml

The first one is train_config_file, which indicates training configurations and the other is compress_config_file, which represents compress configurations. The details of available configurations are described in Here.

After you run DA2Lite to compress a DNN model, logging and compressed model are saved in ./log directory.

The following shows the format of saving:

• YYYY-MM-DD.HH.MM.SS : format of saved directory for an instance.
  • models
    • origin_{dataset}_{model}.pt : The original model is saved.
    • compress_1_{dataset}_{model}.pt : The first compressed model is saved.
    • ...
  • process.log : The inevitable log is only logged.
  • specific_process.log : The training procedure log is added with process.log

Example

• Run the CIFAR10 example with resnet18 using tucker decomposition.
  • The pretrained-model are decomposed and right after fine-tuned: Here

Result

Cifar10 dataset

Model	Acc(%)	Param num(M)	MACs(G)	File size(MB)	Download
ResNet18	94.74% -> 94.14% (-0.6)	11.17M -> 0.75M (14.81x)	0.56G -> 0.19G (2.96x)	42.70MB -> 2.96MB (14.44x)	Here
Vgg16	90.83% -> 88.37% (-2.46)	14.72M -> 0.38M (39.12x)	0.31G -> 0.1G (3.29x)	56.16MB -> 1.45MB (38.71x)	Here
Vgg16_bn	93.22% -> 92.74% (-0.48)	14.73M -> 0.71M (20.7x)	0.31G -> 0.11G (2.93x)	56.25MB -> 2.77MB (20.29x)	Here

TODO

• Multi-GPU training
• PyTorchMobile conversion
• Train a model based on a custom dataset
• Rand-augmentation for improving an accuracy
• Make a model zoo
• Up-to-date model architectures.
• Train a model for object detection tasks (further future...)
• Compression methods for object detection tasks (further future...)

License

The entire codebase is under MIT license