Giters

fcdl94 / MiB

Official code for Modeling the Background for Incremental Learning in Semantic Segmentation https://arxiv.org/abs/2002.00718

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

Difference in the mean IoU on test dataset

Sreeni1204 opened this issue · comments

commented

Hello,

Thanks for the good work.

I have a few questions to ask.

  1. Is the step "1-15" under task "15-5" and step "1-15" under task "15-5s" the same? If yes, why is there a difference in the performance or mean IoU values in the paper?

  2. I performed the FT and MiB experiments, and the mean IoU is differing too much from the one in the paper. The difference between the command given by you and the one I am using is that I have a single CUDA device and reduced the batch size to 12.

Thanks and regards,
Sreeni...

commented

Hello @Sreeni1204,

  1. Yes, the model is the same. However, the performance is reported after the last incremental step is done. Thus, doing one incremental step or doing multiple give indeed different results.
  2. Well, if you reduce the batch size, try to decrease also the learning rate (our empirical rule is to divide the LR by the same rate - in your case, halve it). Remember also that experiments in the 15-1 scenario used a loss_kd value of 100, while for all the other settings we used 10.

Sorry for the late answer.