Hyperparameters differences between code and paper
Piotr94 opened this issue · comments
Hi,
I am trying to reproduce results from your paper and I have noticed that there are some differences between hyperparameters described in the paper and these in the official code:
number of patches (384000/256000),
batch size (96/64),
maximum noise value during training (50/55),
scaling for augmentation (it's mentioned in the paper, but I think it isn't implemented in the current version of the code, at least I cannot find it).
Do you know about any other differences? Could you share how you implemented scaling for the augmentation of the training data?
Also I have made some experiments with different version of your repository and it seems to me that version with old DALI dataloader (v.0.1) gives significantly better results (on validation it was 0.5 dB).
Hi,
From what I understand of your question, what you're looking for is in the function normalize_augment() in utils.py, #57. The chances of executing each augmentation, w_aug, are defined inside this function.
In the training loop, the augmentation function is called after getting a training example from the Dali dataloader:
img_train, gt_train = normalize_augment(data[0]['data'], ctrl_fr_idx).
I can't think about other differences in parameters apart from the ones you mentioned.
Regarding DALI versions, I suspected that the new one came with a drop in performance. Thank you very much for confirming this (btw do you mean it was 0.5db higher in validation with v.0.1?).
Hope this helps. Let me know if you have additional question.
Version of repository tagged as 0.1 gave me better validation score by 0.55 dB. The only significant change in my opinion is change of dataloader but also other libraries were updated and there is small change in utils.py.
In normalize_augment there are methods for rotating and flipping frames, also one for adding random scalar to images but there is nothing for scaling.
I see, thanks for your feedback.
As for the augmentations, no other random transformations apart from those in normalize_augment() and the Dali dataloader (random crops) were applied.