A question about training with RGB images

[Kaixuan Wei]

Hi, nice work. I look at the example 2D and find you augment data with shape N x H x W x 1 by a mask such that the final input size would be N x H x W x 2. When it applies to RGB image, is that mean we should augment the input image size to N x H x W x 6? Thank you in advance.

[Florian Jug]

@alex-krull @tibuch ?

[Tim-Oliver Buchholz]

Hi @Vandermode
Apologies for the delayed answer.

The input (X) should just be the normal image i.e. N x H x W x 1 for grayscale and N x H x W x 3 for RGB images. We decided attach the mask to the target (Y), since the mask is only used during training. In the case of a RGB image 3 additional masks are needed, because every channel gets its own mask

n2v/csbdeep/internals/train.py

Line 158 in 7a950bc

 self.Y_Batches[(j, *coords[k], self.n_chan+c)] = 1 

. This is possible because the noise is independent across channels.

It just occurred to me that our current implementation is not optimal. We mask the same pixel in all channels

n2v/csbdeep/internals/train.py

Lines 155 to 159 in 7a950bc

 for k in range(len(coords)): 
     for c in range(self.n_chan): 
         self.Y_Batches[(j, *coords[k], c)] = y_val[k][c] 
         self.Y_Batches[(j, *coords[k], self.n_chan+c)] = 1 
         self.X_Batches[(j, *coords[k], c)] = x_val[k][c] 

, which is not required. I will investigate and fix this issue.