The RMI loss does not change too much
xychenunc opened this issue · comments
I tested the rmi loss with random inputs and I found the rmi loss does not change too much. Is it normal? My test code is as follows.
logits = np.random.randn(5, 3, 32, 32)
labels = np.random.randint(0, 3, size=(5, 32, 32))
logits = torch.from_numpy(logits.astype(np.float32))
labels = torch.from_numpy(labels.astype(np.int32))
rmiloss= RMILoss(num_classes=3)(logits, labels)
print(rmiloss)
What does the "rmi loss does not change too much" mean?
Besides, rmi tries to model the relationship between pixels in images. The order of the pixels is important and meaningful, while the random inputs are disordered.
I tested the RMI part with random inputs and I found the value RMI loss is around the same value. The actual values do change, but the change is pretty small if compared with loss value itself.
Now I tested the loss function with real dataset and I did not see improvement. The performance of model with RMI loss is almost the same as my baseline model. In my experiments, I assign CE loss with weight 0.9 and RMI loss 0.1. I am wondering if your proposed RMI loss is sensitive to loss weights. And, by the way, if it is sensitive to rmi_radius. Thanks.
Iforgot to mention that when I tested the code, I removed the downsampling (using strided convolution or interpolation) part.
I tested the RMI part with random inputs and I found the value RMI loss is around the same value. The actual values do change, but the change is pretty small if compared with loss value itself.
Now I tested the loss function with real dataset and I did not see improvement. The performance of model with RMI loss is almost the same as my baseline model. In my experiments, I assign CE loss with weight 0.9 and RMI loss 0.1. I am wondering if your proposed RMI loss is sensitive to loss weights. And, by the way, if it is sensitive to rmi_radius. Thanks.
In fact, we also tested RMI with different weights, results on PASCAL VOC val set with DeepLabv3 are: {0.1, 0.3, 0.5, 0.7, 0.9} – {77.49, 77.88, 78.71, 78.50, 77.40}(%, mIoU).
{01, 0.9} yield roughly same mIoU as baseline. lambda=0.5 is our first and best choice.
Additionally, using sigmoid operations when using RMI, as we discussed in the paper.
About the downsampling, indeed, we do not test the case without downsampling. I am not sure if there are any other underlying problems. I am also glad to know your results.