No such file or directory: 'mmseg/utils/sampler/city_lbl2idx.pth'

[Fang (Kevin) Li]

hi, when i try to reproduce your work for retraining on cityscapes dataset. It seems like I still miss one checkpoint. Can you help me to tackle this problem? thank you!

[whiteinblue]

the same question ? how to generate file of xx_lbl2idx.pth ?

[Leon Liang (Chen Liang)]

Sorry for the confusion and inconvenience caused. In this work, we assume a uniform prior for class probabilities, but in practice, segmentation datasets like ADE20K, Cityscapes, and COCO-Stuff have long-tailed pixel distributions.

To address this issue, we propose a pixel balanced sampler to rebalance the pixel distribution as a uniform distribution. You can find the relevant code for the sampler in the following link:

GMMSeg/mmseg/utils/sampler.py

Lines 42 to 95 in da1a518

 class BatchBalanceClassSampler(Sampler): 
  
     def __init__( 
         self, 
         dataset, 
         cfg 
     ): 
         """Sampler initialisation.""" 
         super().__init__(dataset) 
         logger = get_root_logger() 
         logger.info(f'Enable Sampling Mode [BALANCE] ') 
         self._num_classes = 1 # * one class per batch 
         self._batch_size = 1 
         self._num_batches = len(dataset) // self._batch_size 
         self._labels, self.lbl2idx = self.gather_labels(dataset) 
          
     def gather_labels(self, dataset): 
         num_labels = len(dataset.CLASSES) 
         labels = list(range(num_labels)) 
  
         _dataset_dict = { 
             # * only compatible with ADE20K, Cityscapes and COCO-Stuff 
             # TODO hard-coded: ugly, may fix 
             150: 'mmseg/utils/sampler/ade_lbl2idx.pth', 
             19:  'mmseg/utils/sampler/city_lbl2idx.pth', 
             171: 'mmseg/utils/sampler/cocos_lbl2idx.pth', 
         } 
  
         lbl2idx = torch.load(_dataset_dict[num_labels]) 
  
         return labels, lbl2idx 
  
     def __len__(self) -> int: 
         """ 
         Returns: 
             number of samples in an epoch 
         """ 
         return self._num_batches 
  
     def __iter__(self) -> Iterator[int]: 
         """ 
         Returns: 
             indeces for sampling dataset elems during an epoch 
         """ 
         indices = [] 
         for _ in range(self._num_batches): 
             cls_id = random.sample(self._labels, self._num_classes)[0] 
             replace_flag = self._batch_size > len(self.lbl2idx[cls_id]) 
             batch_indices = np.random.choice( 
                 self.lbl2idx[cls_id], self._batch_size, replace=replace_flag 
             ).tolist() 
             indices.append(batch_indices[0]) 
  
         return iter(indices) 

Regarding the xx_lbl2idx.pth files, they contain the mapping of each label to pixel indexes within the corresponding label class.

We are currently working on releasing both the code for generating these files and the files themselves. Please stay tuned for updates.