alievk / npbg

Hi, thank you for the great work! I have a doubt about the code. During training, you have set eval mode to true. I'd be grateful if you could help me understand why that's so and whether there is a difference in inference results if we set the model to train mode instead.

Thanks in advance

Hi @Shubhendu-Jena, thank you for appreciating our work!
Are you sure that we actually set eval mode to True? As far as I can see, we set the train mode to True before the epoch starts:

npbg/train.py

Line 247 in 355294b

pipeline.model.train()

Or perhaps I'm missing something?

Hi,

Thanks for the quick response. Actually, it doesn't go to that line as args.eval_in_train evaluates to True in

npbg/train.py

Line 241 in 355294b

    
           if args.eval_in_train or (args.eval_in_train_epoch >= 0 and epoch >= args.eval_in_train_epoch):

This is because in train_example.yaml, we have the following:

npbg/configs/train_example.yaml

Line 6 in 355294b

eval_in_train: True

Ah ok, if you use eval_in_train: True in the train config, it'll indeed use the eval mode in train. Basically, turning this option off would enable the normal train mode.
Let me see if it's better to remove this line from our train config example.

Just to clarify, the numbers reported in the paper are with train mode, right? And have you observed any difference in the evaluation results with train mode vs eval mode?

I think we only used train mode for the results in the paper; @alievk do you remember by any chance?

Eval mode=True during training