pos_weight for segmentation task
Kaiseem opened this issue · comments
Hello, you are doing a great job. I have some questions and I hope to get your response.
I notice that there is a pos_weight in the uncertainty loss (nn.BCEWithLogitsLoss) but it seems not to be detailed in the paper. In particular, it is 0.01 in the toy example and 5 in the deep estimation task. I want to know how to determine this parameter? and what you set in the segmentation task?
Thank you for your comment! We will try to add it to the paper. To be honest, it's a super hard parameter to tune like in the new ImageNet training protocole. To do this, we set it to 5 for semantic segmentation, but we visualized the results on the validation set to tune this parameters. Here is how we set it.
Thanks for your response, it is helpful. Meanwhile, would you like to share the segmentation code? If so, it is quiet helpful for others to follow.
@Kaiseem Could you reproduce the semantic segmentation task?
@yaoyuan10475 i didn't reimplement the semantic segmentation in this task line, but directly apply it to a novel task, https://dl.acm.org/doi/abs/10.1145/3583780.3614872
From my experiences, pos_weight need to be tuned carefully when applied to a new task.
Thank you for your reply. For semantic segmentation tasks, if I only have two categories: target class and background, how should the number of nb Prototypes be adjusted?
@yaoyuan10475 For your reference, in my task there are 19 classes (Cityscapes), and i set 64 prototypes with dimensions of 256.
This is very helpful to me.
I have another question. Does this setting take up a lot of GPU memory?
@yaoyuan10475 nope, only a litttle additional GPU memory is required
Thank you for your patient reply. When I follow your settings, an out of memory error occurs. My feature layer dimensions are [12, 256, 128, 128] (channel order is [B, C, H, W]), nb_prototypes is 64, and the dimension is 256. An error will occur. When I changed the feature layer dimensions to [12, 256, 64, 64], the GPU memory usage was reduced a lot. Is it because the length and width of the feature layer are too large? Or is it because of other reasons?
I took out the code separately, as follows.
import torch
import torch.nn.functional as F
from torch import nn
class Distanceminimi_Layer_learned(nn.Module):
def __init__(self, in_features=0, out_features=0):
super(Distanceminimi_Layer_learned, self).__init__()
self.in_features = in_features
self.out_features = out_features
self.omega = nn.Parameter(torch.Tensor(1, out_features, in_features, 1, 1))
self.reset_parameters()
def reset_parameters(self):
nn.init.normal_(self.omega, mean=0, std=1)
def forward(self, x):
x = x.unsqueeze(1)
out = F.cosine_similarity(x, self.omega, dim=2, eps=1e-30)
return out, self.omega
if __name__ == '__main__':
DM = Distanceminimi_Layer_learned(256, 64)
DM.to("cuda:0")
feat = torch.randn(size=(12, 256, 128, 128)).to("cuda:0")
out, omega = DM(feat)
print(out.shape)
Your code seems ok from my side, your batch size is a lot bigger than ours, which caused the out-of-memory issue. One solution could be mixed-precision training.
According to your suggestion, out-of-memory problem has been solved.
Thank you very much for your help. @Kaiseem @xuanlongORZ @giannifranchi