how to deal with 'difficulty' label
bryan-bisonai opened this issue · comments
I wonder how the evaluation script for test set deal with the "difficulty" labels. As written in the DOTA_devkit/dota-v1.5_evaluation_task2.py
script code when difficulty is labeled as 1, which means the instance is difficult to be detected due to blur or too small, the instance is discarded although no penalty is incurred for detecting them.
Link to the code:
DOTA_devkit/dota-v1.5_evaluation_task2.py
Line 143 in 4db7aed
I would like to ask if this rule will be applied on the CVPR 2019, Detecting Objects in Aerial Images, competition as well.
CVPR 2019 is different from DOTA-v1.0. Please see the notes in https://captain-whu.github.io/DOAI2019/evaluation.html.
So according to the statement in the link you provided, it says Note, in the evaluation, the instances with a difficult label is also included, which is different from the DOTA-v1.0.
So... what is the main difference of a difficult label between DOTA-v1.0 and DOTA-v1.5?
In the annotation of DOTA-v1.5, many small instances(even below 10 pixels) are also labeled.