A question about nuscences test set result
ged0606 opened this issue · comments
Hello, thanks for your work.
I have a question about nuscenes test set result listed in this paper
The mAP of FMF-Voxelnet-Base seems wrong, I get 64.6 according per-class AP.
We have updated the baseline code. There is a little improvement in the metrics (at least on Waymo). How many sweeps do you use?
Thanks for your reply. Sorry that my expression is not clear. I have not implement the code, just read the paper list above.
I compute mAP by per class AP listed in the paper, just like:
(89.8 + 59.3 + 67.0 + 68.6 + 32.8 + 85.7 + 58.6 + 30.2 + 77.4 + 77.0) / 10 = 64.64.
I am using Nuscenes-devkit to calculate the metrics. I will check if weighted average is implemented. Otherwise, I will updated the metrics in a new arxiv version.
Thanks for highlighting this issue.
Correct. Thanks for mentioning this mistake. There is a typo in the table.