For multiclass, how to get the overall score?
sarmientoj24 opened this issue · comments
Do I just average all of it?
Heyho @sarmientoj24,
the majority just macro-averages it (computing the score for each class individually and then just average it).
So in terms of miseval, you get the score for each class if you run the evaluate function with the multi_class parameter, and the you can just apply np.mean() from NumPy on the returned array.
For the evaluation biases in multi-class, check out the subchapter "Multi-class evaluation" in our recent paper in BMC Research Notes Towards a guideline for evaluation metrics in medical image segmentation: https://bmcresnotes.biomedcentral.com/articles/10.1186/s13104-022-06096-y
Also check out this interesting StackExchange post about Micro- vs Macro-Averaging: https://datascience.stackexchange.com/questions/15989/micro-average-vs-macro-average-performance-in-a-multiclass-classification-settin
Cheers,
Dominik
how do i micro average here?
@sarmientoj24, you have to put some more information to your questions 😄
sorry for that. i was trying to ask how do I compute micro-averages instead of macro averages using the library to get the score?
@sarmientoj24, MISeval does not provide functionality for automatic computing of averaged values. You can get the individual class scores from MISeval, but you have to do the averaging/weighting by yourself as you want. You can compute the class distribution per sample and then accordingly average the miseval returned scores with your weights.