Want to share with you the result on NuScenes validation set here. I have not changed any setting or configuration. Centerpoint input. 2Hz
BaiLiping opened this issue · comments
Per-class results: | |
---|---|
AMOTA AMOTP RECALL MOTAR GT MOTA MOTP MT ML FAF TP FPFN IDS FRAG TID LGD | |
bicycle 0.414 1.161 0.478 0.846 1993 0.400 0.160 39 67 10.0 943 1451041 9 8 1.59 2.02 | |
bus 0.721 0.766 0.755 0.957 2112 0.718 0.388 60 13 4.3 1584 68518 10 17 1.40 1.91 | |
car 0.637 0.828 0.700 0.935 58317 0.636 0.241 1714 835 44.6 39656 2564 17504 1157 1019 0.97 1.56 | |
motorcy 0.524 1.013 0.583 0.907 1977 0.516 0.243 36 25 7.7 1124 104824 29 29 1.69 2.35 | |
pedestr 0.688 0.737 0.773 0.873 25423 0.654 0.291 983 292 55.4 19049 2424 5778 596 466 0.63 1.14 | |
trailer 0.384 1.314 0.506 0.711 2425 0.352 0.526 44 56 32.5 1201 3471199 25 31 1.62 2.45 | |
truck 0.475 1.104 0.554 0.822 9650 0.446 0.323 149 192 25.1 5238 9304300 112 120 1.70 2.67 | |
Aggregated results: | |
AMOTA 0.549 | |
AMOTP 0.989 | |
RECALL 0.621 | |
MOTAR 0.865 | |
GT 14556 | |
MOTA 0.532 | |
MOTP 0.310 | |
MT 3025 | |
ML 1480 | |
FAF 25.7 | |
TP 68795 | |
FP 6582 | |
FN 31164 | |
IDS 1938 | |
FRAG 1690 | |
TID 1.37 | |
LGD 2.01 |
@BaiLiping I have updated my inference files, please refer to README for the dropbox links.
I run SimpleTrack and the performance is correct. In your case, maybe you need to check if you use the correct config file --configs/nu_configs/giou.yaml
. For example, my script is python tools/main_nuscenes.py --name cp_sanity --config_path configs/nu_configs/giou.yaml --det_name cp --data_folder /mnt/truenas/scratch/ziqi.pang/datasets/nuscenes/validation_2hz --process 150
.
However, I think figuring out the errors will be a lot easier with the inference file I just provided. Good luck!
Got it. Thanks a lot!
i think i know why the result is so different now. The inference file is generated using only 1 lidar sweep right? the inference file i used is generated with 10 aggregated lidar sweeps.
@BaiLiping It depends on the detection file you use. The strongest center point uses 10 sweeps, which is my inference file. Maybe yours uses 1 sweep?
mine is 10 sweeps as well. I will have another try with the inference file you provided. Thanks a lot.