NVIDIA / semantic-segmentation

Hello,
Thank you for your work.

I am trying to figure out if there is a problem with RandomSizeAndCrop and centroids, because in __getitem__ of the base_loader some images centroid is None. I can't figure out why because all image have at least 1 centroid.

My dataset have 1024x1024 images with 3 classes (0-background, 1 - class_1, 2- class_2). I just have 300 images.
class_1 and class_2 are very small (all the masks have at least 1 of the classes and majority of the image is background), so I have a class imbalance problem.
I am build_centroids with 512x512 tile, I check the json file and looks fine (all the images have at least 1 centroid for 1 of the classes). I just did centroids.pop(0) (inside build_centroids, when cfg.GLOBAL_RANK=0) I don't need background centroids, because I want data augmentation in class_1 and class_2.

My mydataset_cv0_tile512.json looks fine, at least 1 centroid per image, belonging to class_1 or class_2.
I did dump_augmentation_images to take a look at the augmentation images. So I have some images without my class_1 or class_2, that is not good because i am adding an image without any class just background class. When I check the mydataset_cv0_tile512.json that image have centroid but __getitem__ of the base_loader is saying centroid is None.
Can you help me on this?
Thanks.

mydataset_cv0_tile512.json short example:

{
    "1": [
        [
            "/dl/1-hierarchical/data/mydataset/images/119_oooo.png",
            "/dl/1-hierarchical/data/mydataset/masks/119_oooo_gt_labelIds.png",
            [
                887,
                791
            ],
            1
        ],
        [
            "/dl/1-hierarchical/data/mydataset/images/4_pppp.png",
            "/dl/1-hierarchical/data/mydataset/masks/4_pppp_gt_labelIds.png",
            [
                893,
                801
            ],
            1
        ],
        [
            "/dl/1-hierarchical/data/mydataset/images/44_iiii.png",
            "/dl/1-hierarchical/data/mydataset/masks/44_iiii_gt_labelIds.png",
            [
                818,
                413
            ],
            1
        ],
        [
            "/dl/1-hierarchical/data/mydataset/images/44_iiii.png",
            "/dl/1-hierarchical/data/mydataset/masks/44_iiii_gt_labelIds.png",
            [
                776,
                514
            ],
            1
        ]
    ],
    "2": [
        [
            "/dl/1-hierarchical/data/mydataset/images/119_oooo.png",
            "/dl/1-hierarchical/data/mydataset/masks/119_oooo_gt_labelIds.png",
            [
                550,
                691
            ],
            2
        ],
        [
            "/dl/1-hierarchical/data/mydataset/images/215_uuuu.png",
            "/dl/1-hierarchical/data/mydataset/masks/215_uuuu_gt_labelIds.png",
            [
                331,
                594
            ],
            2
        ]
    ]
}

This is my script:

HPARAMS: [
  {
   dataset: mydataset,
   cv: 0,
   syncbn: true,
   apex: true,
   fp16: true,
   crop_size: "500,500",
   bs_trn: 4,
   poly_exp: 2,
   lr: 5e-4,
   rmi_loss: true,
   max_epoch: 150,
   n_scales: "0.5,1.0,2.0",
   supervised_mscale_loss_wt: 0.05,
   snapshot: "ASSETS_PATH/seg_weights/ocrnet.HRNet_industrious-chicken.pth",
   arch: ocrnet.HRNet_Mscale,
   result_dir: LOGDIR,
   RUNX.TAG: '{arch}',
   freeze_trunk: true,
   class_uniform_pct: 0.3,
   class_uniform_tile: 512,
   start_epoch: 0,
   do_flip: true,
   dump_augmentation_images: true,
  },
]

I figure it out, is because of this line:

semantic-segmentation/datasets/uniform.py

Line 304 in 7726b14

imgs_uniform = random_sampling(imgs, num_rand)

So here you are random sampling the images without taking into account the centroids or classes.
In the next lines of code you're doing random sampling based on centroids per classes, who is great to uniform sampling the classes.

I don't understand why you are doing the step 1. Why?

I propose to write that line of code here, because it means class_uniform_pct = 0.0

semantic-segmentation/datasets/uniform.py

Lines 317 to 318 in 7726b14

    
           if centroid_len == 0: 
        
               pass

Maybe I am wrong, but what you think about that?

Thank you.

Data Augmentation (RandomSizeAndCrop) - centroids issue