mAP becomes low after merging files.
mamunir opened this issue · comments
Checked resultsmerge and dota evaluation task 2 python files, they both seems legit. Is it normal, I do not think so. After merging, it should increase? On crop images, I have mAP=40 approx but after merging, I got mAP=15 approx.
I set source file in result merge to be 15 files
destination file, where it should be saved after merge and nms
For dota evaluation task 2,
detpath = r'PATH_TO_BE_CONFIGURED/Task1_{:s}.txt' --> destination file, where it should be saved after merge and nms
annopath = r'PATH_TO_BE_CONFIGURED/{:s}.txt' --> val/labelTxt
imagesetfile = r'PATH_TO_BE_CONFIGURED/valset.txt' --> text file path for original large img size validation set
Please see my settings and comment. Thanks
It is not normal. I am not sure of the problem. As a suggestion, you can visualize the predicted bounding boxes on images(for both crop images and large size images), and see if it is correct.
I have stopped after that day. Resumed it now. I have seen by drawing the boxes after merging on respective images. Boxes become wide as compared to cropped images. In cropped images, they were tighter but in original images, they become loose. I am looking into code again, how it handles the variations like translation and mapping etc. Let me know, if you understand the problem. This looks like weird problem
What is the scale(0.5, 1 or 2?) you set when crop or merge images?
rate is set to 1. I see in code, this refer to scale.
Could you provide your results? And give a detail operation description. If so, I will try to reproduce your results.
sure.
am i supposed to send on jding[at]whu[dot]edu[dot]cn?
Yes, and attach a detailed description of your operation.
okay. thank you for your time
sent, please check your email
@mamunir I did not receive it yet. You can try jianding101@gmail.com
maybe in spam. sent on gmail also.
as a general question, mAP always lower even a little when merged or always higher than cropped when merged? or random?
It seems that you use oriented labels. However, to calculate mAP for Task2, we need to use the horizontal labels. We have uploaded the Task2_gt in Baidu drive before. Now we also add the Task2_gt in Google drive. You should use Task2 version.
Yes you are right, problem lies here. I also sorted it just, see your input now. What I did, make min max of x points and y points out of oriented labels. Now from 40.25 mAP, it degrades to 38.5 rather than 15.9 previously. This seems normal to me, good?
I have not tested on crops before. But in my opinion, the difference between crops and large images is caused by the instances on the edge. There may be only part of an instance appear on crops. In the generated ground truth of crops, if the proportion is below 0.7, we give a difficult label. You can try to ajust the paramater or more models and verify relationship between crop results and large image results.
Thank you :)