Table 2 F_Beta Scores don't add up
pshagnea opened this issue · comments
Looking at table two, there's something amiss. Under the first heading, "Non-Parametric w/ Pruning (p = 0.13)", the Precision and Recall scores are equal but the F_0.5 score is 0.71. If the precision and recall are equal the score should also be the same, no matter what the Beta is.
Thresholding Approach Precision Recall F0.5 score
Non-Parametric w/ Pruning (p = 0.13)
MSL 92.6% 69.4% 0.69
SMAP 85.5% 85.5% 0.71
Total 87.5% 80.0% 0.71
I calculated the F_0.5 scores for a few other rows in the table based off of the precision and recall and was unable to get the same resulting score as the paper. Am I interpreting this table correctly?
Good catch, others have found this as well - see answer in #36
Thanks for addressing it!
Re-calculating the F_beta score with the precision and recall, the performance is higher than reported in the paper.
Precision Recall F_0.5_original F_0.5_New
Non-Parametric w/ Pruning (p = 0.13)
MSL 0.926 0.694 0.69 0.87
SMAP 0.855 0.855 0.71 0.86
Total 0.875 0.8 0.71 0.86
Non-Parametric w/out Pruning (p = 0)
MSL 0.758 0.694 0.61 0.74
SMAP 0.43 0.928 0.44 0.48
Total 0.489 0.848 0.47 0.53
Gaussian Tail (ϵnorm = 0.0001)
MSL 0.842 0.444 0.54 0.71
SMAP 0.885 0.783 0.71 0.86
Total 0.875 0.667 0.66 0.82
Gaussian Tail (ϵnorm = 0.01)
MSL 0.613 0.528 0.48 0.59
SMAP 0.824 0.812 0.68 0.82
Total 0.758 0.714 0.62 0.75
Gaussian Tail w/ Pruning (ϵnorm = 0.01,p = 0.13)
MSL 0.882 0.417 0.54 0.72
SMAP 0.927 0.739 0.71 0.88
Total 0.917 0.629 0.66 0.84