LassoNet for Multitask Learning
pranavsankhe opened this issue · comments
Can lassonet be extended to multitask learning neural networks? If so, how do I go about implementing it? TIA
Hi, thank you for this very interesting question!
Multitask learning involves sharing weights from the first layers, then specializing them for different tasks.
In LassoNet, there is a skip connection with a L1 regulatization linking the input to the output. If you want the masks to be the same for different tasks, you could first try a multidimensional regression.
There might be gain to split the architecture at some point, which would involve some smart handling of the dimensions in model.py