Giters

tianyic / only_train_once

OTOv1-v3, NeurIPS, ICLR, TMLR, DNN Training, Compression, Structured Pruning, Erasing Operators, CNN, Diffusion, LLM

Home Page:https://openreview.net/pdf?id=7ynoX1ojPMt

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

some confuse about Algorithm-3

gaoxueyi0532 opened this issue · comments

commented

please explain the function Lines 9-10 of Algorithm-3 in detail, thanks very much!

commented

Thanks for the question. In short, line 9-10 is a Half-Space projector (Eq 9) proposed in the HSPG paper https://arxiv.org/abs/2009.12078 with more rigorous theoretical guarantee in a following theoretical paper https://openreview.net/pdf?id=KBhSyBBeeO.

image

Its general goal is to effectively project a group of variables onto zero (produce group sparsity) and avoid objective function value regression as much as possible.