Comparison with cv.glmnet(alpha=1)

[Uwaise Ibna Islam]

I am looking into the mean cross-validation error of the best model selected in LassoNet with (M set to 0.0) and hidden_dims=(1,) and cv.glmnet() [documentation: https://www.rdocumentation.org/packages/glmnet/versions/1.6/topics/cv.glmnet ] with the same set of lambda values for a classification problem. However, they do not yield similar results. The parameters used are exactly as follows:

lambdas = (0,0.00001,0.0001,0.001,0.01,0.1,1,10,100,1000,10000)

LassoNet=

LassoNetClassifierCV(hidden_dims=(1,),M=0.0,random_state=0,lambda_seq=lambdas,torch_seed=0,cv=LeaveOneOut())

Glmnet=

cv.glmnet(X, Y, family = "binomial", alpha = 1, lambda =lambdas, type.measure = "class", nfolds = 34) [there are 34 instances in the dataset, so nfolds=34 is same as LeaveOneOut()]

If you could explain why these two are behaving differently, it would be really helpful. Also, do you consider the matrix multiplication of skip.weight and layer.weight of the output layer equivalent to feature coefficients in the logistic regression with lasso penalty?

[Louis Abraham]

Hey that's perfectly normal because... LassoNet is not Lasso.

You could try https://scikit-learn.org/stable/modules/generated/sklearn.linear_model.LogisticRegression.html with a L1 penalty.

[Uwaise Ibna Islam]

I understand that the lassonet is not the same as the lasso, but if I want to force Lassonet to behave as a logistic regression classifier with L1 penalty what will be the constraints I need to apply on Lassonet?

[Louis Abraham]

Set M=1e-10 (I'm not sure that M=0 will not produce an error). However even if the models will be the same, the optimization is not so you will not reproduce the same coefficients.