cannot reproduce DA results from Table I in the paper

[Alexey Abramov]

Hi,

I'm trying to reproduce results of the Domain Adaptive Faster R-CNN for GTA Sim 10k -> Cityscapes domain shift (last row in Table I in your paper). However, I'm getting much lower numbers somehow.

My training data looks as follows (trainval.txt file):

source_3384639
source_3384643
source_3384645
...
target_aachen_000000_000019_leftImg8bit
target_aachen_000001_000019_leftImg8bit
target_aachen_000002_000019_leftImg8bit
...

source_* are GTA Sim 10k images from Driving in the Matrix, while target_* are Cityscapes train images (2975). Therefore, 12975 images altogether in trainval. In GTA Sim 10k annotations I have replaced motorbike class by Cityscapes' motorcycle.

The test data (test.txt) consists of Cityscapes val images (500):

target_frankfurt_000001_023369_leftImg8bit
target_frankfurt_000001_075296_leftImg8bit
target_frankfurt_000000_006589_leftImg8bit
...

For the training I'm just following instructions from your GitHub page:

./tools/train_net.py --gpu 0 --solver models/da_faster_rcnn/solver.prototxt --weights data/imagenet_models/VGG16.v2.caffemodel --imdb voc_2007_trainval --iters 70000 --cfg models/da_faster_rcnn/faster_rcnn_end2end.yml

I'm getting the following car AP values:

10k: 32.72
20k: 31.12
30k: 33.51
40k: 33.98
50k: 30.90
60k: 33.53
70k: 33.94

These numbers are much lower as compared to 38.97 from the paper. Do you have any idea what might be wrong in my pipeline?

Thanks a lot in advance for your help! Very much appreciated!

Best,
Alexey

[Yuhua Chen]

Hi Alexey, an example of sim10k to cs is provided in Caffe2, you could check it out:
https://github.com/krumo/Detectron-DA-Faster-RCNN

[xiong233]

Hello, can you reproduce the results now? I have encountered similar problems. Do you use all the categories of SIM 10k or only one category of car when training the source domain?