Want to drop the data and prediction columns in the explain dataframe output
busintelanalytics opened this issue · comments
Hi Thomas, great package.
Aloha from Hawaii!
I'm attempting to process 50K cases, 4 features per case. I'm really only after the content below, minus the data and prediction columns, since these columns explode the dataframe size. How do I run the explain function without outputting the data and prediction columns?
A data.frame encoding the explanations one row per explained observation. The columns are:
• model_type: The type of the model used for prediction.
• case: The case being explained (the rowname in cases).
• model_r2: The quality of the model used for the explanation
• model_intercept: The intercept of the model used for the explanation
• model_prediction: The prediction of the observation based on the model used for the explanation.
• feature: The feature used for the explanation
• feature_value: The value of the feature used
• feature_weight: The weight of the feature in the explanation
• feature_desc: A human readable description of the feature importance.
• data: Original data being explained
• prediction: The original prediction from the model**
Currently you can't... But bear in mind as well that explaining 50k observations is a crazy endeavour as the processing time will amount to days... The lime algorithm has never really been designed for such use
if you are running it in batches you could simply drop the columns manually, after you get the explanation for each batch... It will be some time before I get to making further work on lime