How to Access LassoNet Model Wieghts
imessien opened this issue · comments
I'm trying to pull the model weights selected in the final optimized regularization path for LassoNet. Is there anyway to show the weights for each of the features used in that path? I know you can print out the most complex path using path[-1] but how do I access the weights there? I've tried using path[-1].state_dict() but that didn't work. Is there a special way to call the weights in the path? I know you can use model.feature_importance_ to show the feature importance but that doesn't serve as the weight in the model. Correct? The model.coef_ attribute doesn't work just to print out the weight. Could you please provide some guidance on how to obtain the weights or coefficients of the LassoNet model? Thanks for your help.
Can you provide the full code? The answer is not the same depending on the method you called. For a CV model, you do model.path_[-1].state_dict.
I was using the LassoNetRegressorCV. Thank you for the suggestion it worked that way. My goal is to create a scatter plot with all the individual features and show the final optimal path taken by connecting them. I want to annotate the top 10 features on this line. I want to shade the area underneath the line to show the selected features region. This area would represent the selection of features most likely used in the model but not the top 10. The points above the line weren't included in the final path taken by the model. One axis would be weights and I can't figure out what the other one would be. I think accuracy would be with respect to the output of the model. I want to give it weight I'm confused about what my axis should be can it be accuracy and coefficients? When I use lambda for x and coefficients and y it would be a vertical line because the best features all have the same lambda value. What should my axis be and is this possible? I will use coefficients = model.path_[-1].state_dict()['linear.weight'].numpy() to get the weights of each of the features. Is this possible? It's still not working.
`
model = LassoNetRegressorCV(
hidden_dims=(100,),
M=30,
lambda_start=1e-2,
path_multiplier=1.02,
gamma=.1,
verbose=True,
dropout= 0.5,
cv=5,
n_iters=(3000, 500),
patience=(50,5)
)
model.fit(X_train_scaled, Y_train)
path = model.path(X_train_scaled, Y_train)
feature_names = list(X.columns)
coefficients = model.path_[-1].state_dict()['linear.weight'].numpy()
importance = model.feature_importances_.numpy()
feature_importance = sorted(zip(importance, coefficients, feature_names), reverse=True)
feature_importance = pd.DataFrame(feature_importance, columns=['importance', 'coefficients', 'feature'])
feature_importance = feature_importance.head(10)
top10_features = feature_importance['feature']
print("Top 10 features with their importance and coefficients:")
for feature in top10_features:
print(f"{feature}: Importance - {importance[feature]}, Coefficient - {coefficients[feature]}")`
Traceback (most recent call last):
File "/users/iessien/LassoNetCV5.py", line 140, in
coefficients = model.path_[-1].state_dict()['linear.weight'].numpy()
TypeError: 'dict' object is not callable
You should be able to access it with just
model.path_[-1].state_dict['linear.weight'].numpy()
That method gave me an error message. The correct statement is model.path_[-1].state_dict. 'linear.wieght' is not a key for the state_dict. The keys are ['layers.0.weight', 'layers.0.bias', 'layers.1.weight', 'layers.1.bias', 'skip.weight']. So I will use 'layers.0.weight' to get it. Thank you for your help.