Problem of evaluation on temporal settings.

[weimengchuan]

Thanks for yout great work!

I am doing evaluation on the temporal setttings, but obtain lower performance than that you have reported in github.

My command:

bash tools/dist_test_map.sh projects/configs/maptr/maptr_tiny_r50_24e_t4.py ckpts/epoch_24.pth 1
bash tools/dist_test_map.sh projects/configs/maptr/maptr_tiny_r50_24e_bevformer_t4.py ckpts/maptr_tiny_r50_24e_t4.pth 1

My results:

• maptr_tiny_r50_24e_t4.py
  
  My result is 39.2 mAP, which is lower than 51.3.

• maptr_tiny_r50_24e_bevformer_t4.py
  
  My result is 39.7 mAP, which is lower than 53.3.

Another problem:
When I download the pth files, I find that the configuration file name does not correspond to the weight file name (as is shown in My command).
I try to exchange the order of pth files, and run following commands, but mAPs are almost zero.

Your help is greatly appreciated.

[sanzhang]

set "video_test_mode=True"

[会意]

Hi, I encountered the same issue. Follow @iuhiyuh method to modify the configuration model file:
maptr_tiny_r50_24e_t4.py
maptr_tiny_r50_24e_bevformer_t4.py
set "video_test_mode=True"
I could achieve the same mAP as in the README.md