Giters

tianyic / only_train_once

OTOv1-v3, NeurIPS, ICLR, TMLR, DNN Training, Compression, Structured Pruning, Erasing Operators, CNN, Diffusion, LLM

Home Page:https://openreview.net/pdf?id=7ynoX1ojPMt

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

How to compress the backbone of a model with an auxiliary branch？

wangxd15 opened this issue · comments

commented

Very nice work !
I am compressing a model with a backbone and an auxiliary branch, but hope to keep the auxiliary branch unpruned. Does OTO support such operations and how to set? Thanks

commented

@wangxd15 This is a good question.

We have ongoing plan to support it more conveniently in the future releases after completing current working items.

For a short-term fix, you could print out the dependency graph, and find the connected components corresponding to the auxiliary branches. Then manipulate the type for these connected components as GROUP_TYPE['default] which would be skipped from being pruned. In particular, edit the cc_param_groups['group_type'] = GROUP_TYPE['default'] in param_groups() function of Graph class.