Performance on Cityscapes to Cross-City

[Binhui Xie (谢斌辉)]

Hi, thank you for your great work. I'm trying to reproduce the results of Cityscapes to Cross-City, but the performance is not as good as yours. Here are some questions:

In your paper (Table 1), for Cityscapes to Rio task, the mIoUs of Source DeepLab-v2 and AdaptSegNet are 48.2 and 51.6 respectively. But I got mIoU of 44.9 for Source DeepLab-v2 and mIoU of 47.0 for AdaptSegNet. Therfore, there is performance degradation for FADA and FADA w/ self distillation.

The config file of Source DeepLab-v2:

MODEL:
  NAME: "deeplab_resnet101"
  WEIGHTS: 'https://download.pytorch.org/models/resnet101-5d3b4d8f.pth'
  FREEZE_BN: True
  NUM_CLASSES: 19
DATASETS:
  SOURCE_TRAIN: "cityscapes_train"
  TEST: "crosscity_Rio_test"
INPUT:
  SOURCE_INPUT_SIZE_TRAIN: (1024, 512)
  TARGET_INPUT_SIZE_TRAIN: (1024, 512)
  INPUT_SIZE_TEST: (2048, 1024)
SOLVER:
  BASE_LR: 5e-4
  MAX_ITER: 31250
  STOP_ITER: 20000
  BATCH_SIZE: 8
  BATCH_SIZE_VAL: 1

And the config file of AdaptSegNet:

MODEL:
  NAME: "deeplab_resnet101"
  WEIGHTS: "https://download.pytorch.org/models/resnet101-5d3b4d8f.pth"
  FREEZE_BN: True
  NUM_CLASSES: 19
DATASETS:
  SOURCE_TRAIN: "cityscapes_train"
  TARGET_TRAIN: "crosscity_Rio_train"
  TEST: "crosscity_Rio_test"
INPUT:
  SOURCE_INPUT_SIZE_TRAIN: (1024, 512)
  TARGET_INPUT_SIZE_TRAIN: (1024, 512)
  INPUT_SIZE_TEST: (2048, 1024)
  BRIGHTNESS: 0.5
  CONTRAST: 0.5
  SATURATION: 0.5
  HUE: 0.2
SOLVER:
  BASE_LR: 2.5e-4
  BASE_LR_D: 1e-4
  MAX_ITER: 62500
  STOP_ITER: 40000
  BATCH_SIZE: 8

If possible, could you provide the config files and data loader for reproducing the results?

Thank you anyway.

[Haoran Wang]

Hi, thanks for your interest!

The result of baseline and AdaptSegNet on Cityscapes->Cross-City is copied from AdaptSegNet paper. I also find it difficult to reproduce the results of AdaptSegNet with this codebase. As the author of AdaptSegNet doesn't release codes for Cityscapes->Cross-City, it would be highly possible that my implementation is inconsistent with theirs. I would suggest you implement their methods with their codebase and their provided pre-trained weights to reproduce their results.

For your config file, I think the problem might be your INPUT_SIZE_TEST. I would suggest you use same input size for training and testing for same dataset, as the DeeplabV2 would be sensitive to the scales of objects.

I would contact other authors of this paper to see whether configs and pre-trained weights are kept in the server. I will come back to you in 1-2 weeks.

[Binhui Xie (谢斌辉)]

I'm grateful to you for your replay.

[Haoran Wang]

Hi, the cross_city related code could be found in this file: cross_city_codes.tar.gz. It was implemented on a previous version of this project. Unfortunately recently I don't have time to test these code snippets with this repo. Hope it could give you some hints. Best wishes!