Some questions about the evaluation of experiment results in your paper, especially related to AB3DMOT
Sonne-Zhu opened this issue · comments
Thanks for your great work, but I still have a few questions about the experiment part of the paper.
-
in Table II, the results are obtained on kitti training data, how did you get the AB3DMOT results, did you evaluate them yourself by running the open source code of AB3DMOT?
-
If the answer to question 1 is yes, do you still converted 3D tracking results onto a 2D image plane to get the evaluation result?
-
For every tracked object, its BBs IoU with ground truth is required to be above a threshold in order to be considered as a successful match. What was the IoU threshold set in the experiments in Table II, was it 0.5?
-
Is the MOTA result in Table II the average of all 21 sequences on the traing dataset, or just the average of some of theses 21 equences .
-
If the answer to question 2 is yes, have you evaluated the 3d MOTA using different 3d IOU thresholds (0.25 0.5 0.7) on the traing sequences as in the AB3DMOT paper and compared with the AB3DMOT results in its paper?
I am looking forward to your reply, thank you very much!
1 The results are based on our reproduced code.
3 The results are calculated by the official metrics of KITTI.
4 It is the average of some sequences (maybe last 11 seq, I also somewhat not sure, my bad...) from training set, it is different from AB3DMOT.