Single Scale Evaluation Performanace on Cityscapes with `Seg-T-Mask/16`.

[Jitesh Jain]

Hey, thanks for the great work and codebase!

I tried reproducing your results using the vit_tiny_patch16_384 variant of ViT as the backbone. That works pretty well when training on the ADE20K dataset, giving an mIOU = 38.55 as expected.

However, when I trained the same model on the Cityscapes dataset, it gave an mIOU = 72.9, which is also amazing. Also, it would be great if you can add performances on the cityscapes dataset in the README.

I used the same hyperparameters as were released with the original code. I made one change to the label loading strategy of Cityscapes to map the GT class IDs to a range of 0-18 as all other works do.

Not using label mapping will give the index mix-match error during training. So putting it here in case anyone faces the same issue in the future.

Code to be added in segm/data/base.py:

ignore_label = 255
id_to_trainid = {-1: -1, 0: ignore_label, 1: ignore_label, 2: ignore_label, 
                3: ignore_label, 4: ignore_label, 5: ignore_label, 6: ignore_label, 
                7: 0, 8: 1, 9: ignore_label, 10: ignore_label, 11: 2, 12: 3, 13: 4, 
                14: ignore_label, 15: ignore_label, 16: ignore_label, 17: 5, 
                18: ignore_label, 19: 6, 20: 7, 21: 8, 22: 9, 23: 10, 24: 11, 25: 12, 26: 13, 27: 14, 
                28: 15, 29: ignore_label, 30: ignore_label, 31: 16, 32: 17, 33: 18}

def convert_labels(self, labels):
      labels_copy = np.copy(labels)
      for k, v in id_to_trainid.items():
          labels_copy[labels == k] = v
      return labels_copy

Calling the convert_labels function before returning the GT Labels for the cityscapes dataset does the trick.

[Ricardo García]

Hi @praeclarumjj3,

I am not sure if I understand why do you want to add this code in segm/data/base.py. If you downloaded the Cityscapes dataset by using the preparation script segm/scripts/prepare_cityscapes.py, the PNG labels are already written into the disk by using TrainIds (range 0-18). See:

segmenter/segm/scripts/prepare_cityscapes.py

Lines 55 to 59 in 0d5fb33

 def convert_json_to_label(json_file): 
     from cityscapesscripts.preparation.json2labelImg import json2labelImg 
  
     label_file = json_file.replace("_polygons.json", "_labelTrainIds.png") 
     json2labelImg(json_file, label_file, "trainIds") 

Additionally, when you use Segmenter Cityscapes dataloader segm/data/cityscapes.py, it relies on mmseg, which is making use of the previously written labels, and you can see that here, the ignore_label was already set to 255:

segmenter/segm/data/cityscapes.py

Lines 17 to 23 in 0d5fb33

 class CityscapesDataset(BaseMMSeg): 
     def __init__(self, image_size, crop_size, split, **kwargs): 
         super().__init__(image_size, crop_size, split, CITYSCAPES_CONFIG_PATH, **kwargs) 
         self.names, self.colors = utils.dataset_cat_description(CITYSCAPES_CATS_PATH) 
         self.n_cls = 19 
         self.ignore_label = 255 
         self.reduce_zero_label = False 

I might be missing something. Thank you.

[Jitesh Jain]

I see! I had the already downloaded version of cityscapes and didn't run the preparartion script on the images folder. Thanks for pointing that out @rjgpinel!

Let me close the issue.