doubt about the pre-model precision and recall

[关亚勇]

hi，i use the model you released COCO.res50.384x288.CPN snapshot_350.ckpt，set test_subset= True, the num is first 1000, the result is so low
Average Precision (AP) @[ IoU=0.50:0.95 | area= all | maxDets= 20 ] = 0.111
Average Precision (AP) @[ IoU=0.50 | area= all | maxDets= 20 ] = 0.131
Average Precision (AP) @[ IoU=0.75 | area= all | maxDets= 20 ] = 0.119
Average Precision (AP) @[ IoU=0.50:0.95 | area=medium | maxDets= 20 ] = 0.108
Average Precision (AP) @[ IoU=0.50:0.95 | area= large | maxDets= 20 ] = 0.117
Average Recall (AR) @[ IoU=0.50:0.95 | area= all | maxDets= 20 ] = 0.113
Average Recall (AR) @[ IoU=0.50 | area= all | maxDets= 20 ] = 0.132
Average Recall (AR) @[ IoU=0.75 | area= all | maxDets= 20 ] = 0.120
Average Recall (AR) @[ IoU=0.50:0.95 | area=medium | maxDets= 20 ] = 0.108
Average Recall (AR) @[ IoU=0.50:0.95 | area= large | maxDets= 20 ] = 0.121

this is because of released model is underfitting？i try to draw the keypoints model predicted，show result is not correct.

[Yilun Chen]

Did you ever try res50.256x192 model ? If that's OK, then maybe I should re-submit the model.

[关亚勇]

not yet，I will try other models.

[关亚勇]

@chenyilun95 , i try res50.256x192 model and res101.384x288 model , result is similar
Average Precision (AP) @[ IoU=0.50:0.95 | area= all | maxDets= 20 ] = 0.105
Average Precision (AP) @[ IoU=0.50 | area= all | maxDets= 20 ] = 0.119
Average Precision (AP) @[ IoU=0.75 | area= all | maxDets= 20 ] = 0.113
Average Precision (AP) @[ IoU=0.50:0.95 | area=medium | maxDets= 20 ] = 0.131
Average Precision (AP) @[ IoU=0.50:0.95 | area= large | maxDets= 20 ] = 0.064
Average Recall (AR) @[ IoU=0.50:0.95 | area= all | maxDets= 20 ] = 0.101
Average Recall (AR) @[ IoU=0.50 | area= all | maxDets= 20 ] = 0.113
Average Recall (AR) @[ IoU=0.75 | area= all | maxDets= 20 ] = 0.113
Average Recall (AR) @[ IoU=0.50:0.95 | area=medium | maxDets= 20 ] = 0.132
Average Recall (AR) @[ IoU=0.50:0.95 | area= large | maxDets= 20 ] = 0.058

i just run the mptest.py in corresponding folder, not modify the code, whether need to modify part of code to get right result.

[Yilun Chen]

The submitted models should give the same result in the table. I think there must be somthing wrong. Some other guys has tested the res50.256192 model and got the right result. I’m outside now and I’ll check it again maybe in late afternoon.

[关亚勇]

It is not urgent，thanks，i will try to check.

[关亚勇]

@chenyilun95 ，i got the right result， code is correct，close this.