Which attributes to use for the model

[camilovelez]

Hello Jeevesh,

I would like to ask which attributes from the NSRDB are best suited to be used as inputs for the models.

Thank you so much for your attention, my best regards.

[Jeevesh Juneja]

You can use as many features as you want, Camilo. I would say the more the better. Since I am no meteorologist, I would defer from telling you which feature is important and which is not. And will only tell you statistical methods to find it out for yourself.

If you want to reduce training time or model size, you can plot the correlation heat map by first loading the data into a dataframe and then running :

import seaborn as sns
corr = df.corr()
sns.heatmap(corr, 
            xticklabels=corr.columns,
            yticklabels=corr.columns)

[For more on plotting heatmaps : https://stackoverflow.com/questions/39409866/correlation-heatmap ]
Then eliminate features having very low correlation with the feature(GHI) being predicted. But this is only a heuristic way. And we need to be careful as features having zero correlation doesn't necessarily mean that there is no dependence. So do this, only if you want to reduce size desperately.

Another way could be to first train the model on all features , then replace(in the test set) any feature you wish to remove with its mean value(calculated over train set) and check how much accuracy drops compared to the accuracy on the original dataset. If it doesn't drop by much, you can eliminate that feature, and trian on the reduced feature set.

Also, you can check gradients, if the gradient to a feature is near to zero,almost always, then surely that feature is not influencing the loss much, so you can drop it.

I am glad that you found the repository useful. And wish you all the best for your future endeavours !

Yours sincerely