Entropy estimation uncertainty

[shimanman]

Hello, I am reading through your paper and I have a question that I am not sure can be answered:
The model says that it is trained by estimating uncertainty through entropy, may I ask how exactly the uncertainty is estimated through entropy in the modeling process? Can you elaborate on the process including the code implementation part, thank you very much.

[Himashi Amanda Peiris]

Hello, during model learning the entropy is used to capture the uncertainty in the predictions of the segmentation models. We used the masked cross-entropy losses for the first and second segmentation models, and they incorporate the concept of uncertainty through the use of confidence maps which are generated from two critic/discriminator networks.
These maps highlight regions of interest by estimating the uncertainty of the predicted segmentation (If you have a closer look at the model learning pipeline, you will see that we use the second critic to generate a confidence map for the prediction of segmentation model 1, and vice versa). Then, these confidence maps are binarized using a threshold T to create binary masks. This operation is performed to distinguish between regions with high confidence (where the model is more certain) and regions with low confidence (where the model is less certain). The masked cross-entropy loss is then calculated using the binarized confidence maps. This loss is applied to the predicted segmentation output of one model while considering only the regions where the other model is confident.

• For $\Loss^1_{\text{m}}$ (See Eq 13), it calculates the cross-entropy loss for the first segmentation model, where the confidence map from the second critic is used to mask out regions with low confidence.

• For $\Loss^2_{\text{m}}$ (See Eq 14), it does the opposite: it calculates the cross-entropy loss for the second segmentation model, using the confidence map from the first critic to mask out uncertain regions.

The intuition behind using these losses is to guide the segmentation models during training by focusing on regions where the models are more confident. This helps the models learn which regions to trust, improving their performance and robustness. Hope this clears your doubts.

Thanks,
Himashi