Current evaluation is incorrect
kaleidoscopical opened this issue · comments
It is possible that the 30-nearest neighbors share same labels (only 1 prediction). Therefore, it is impossible to calculate the top-5 acc. Please re-evaluate the scripts. Thank you.
Top-k means first n (n <= k) predictions. Having 1 prediction for certain samples is ok.
Then, your code can not reproduce the reported top-5 results in your paper : )
Please let us know the accuracy you got in experiments. We may help investigate.
@kaleidoscopical which top-5 number are you referring to? Thanks.
Hello!
Thanks for sharing your implementation.
I have the feeling that top-5 scores are incorrectly computed here.
The line computes "the sum" of all 5 predictions. Instead I would write
top5 = top5 + (correct.narrow(1,0,5).sum(dim=1) > 0).sum().item()Can you please correct me if I am wrong?