CAPTAIN-WHU / DOTA_devkit

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

how to deal with 'difficulty' label

bryan-bisonai opened this issue · comments

commented

I wonder how the evaluation script for test set deal with the "difficulty" labels. As written in the DOTA_devkit/dota-v1.5_evaluation_task2.py script code when difficulty is labeled as 1, which means the instance is difficult to be detected due to blur or too small, the instance is discarded although no penalty is incurred for detecting them.

Link to the code:

npos = npos + sum(~difficult)

I would like to ask if this rule will be applied on the CVPR 2019, Detecting Objects in Aerial Images, competition as well.

CVPR 2019 is different from DOTA-v1.0. Please see the notes in https://captain-whu.github.io/DOAI2019/evaluation.html.

commented

So according to the statement in the link you provided, it says Note, in the evaluation, the instances with a difficult label is also included, which is different from the DOTA-v1.0.

So... what is the main difference of a difficult label between DOTA-v1.0 and DOTA-v1.5?

In the annotation of DOTA-v1.5, many small instances(even below 10 pixels) are also labeled.