Giters

lasso-net / lassonet

Feature selection in neural networks

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

Group Lasso

pablolotito opened this issue · comments

commented

Hi, we are very thankful for sharing this code, and we would like to apply it to a group lasso problem. We have a classification problem where the inputs are grouped in blocks all of the same sizes.
We want to adapt your code.
In our case, each input node j has k features that we would like to have the same theta_j.
One form of writing our problem is modifying the constraints In equation (2) of the Lassonet paper, just adding another index for W and leaving the rest as it is.
Do you think it's possible to make minor changes to your code to do that? Could you help us with that?

commented

Thanks for reaching out.

There are multiple ways to do this, and we certainly have some ideas about how to take it forward.

It would be great to have a detailed problem formulation, with the specific mathematical equation. That would help clarify the meaning of "In our case, each input node j has k features that we would like to have the same theta_j."for us.

It might also be helpful to check out the GROUP-Hier-Prox algorithm in the paper. The paper uses it for unsupervised learning/reconstruction examples. It may be possible to adapt it for Group Lasso supervised problems.

commented

I tried some formulation of group lasso and it looks like it works.

you can test it with

pip install git+https://github.com/lasso-net/lassonet.git@group-lasso

The usage is simple: just add a groups parameter with indices of each group (you probably cannot get it wrong, I do a strict input validation).

For short, we replace constraints on features with constraints on groups.

commented

Wonderful, it seems to work for what we want to do. We will try with our data.
Thanks a lot !