Is there not a validation set?
guixianjin opened this issue · comments
In this code, it seems that for CIFAR-10, all 50000 examples are used for train_loader, and 10000 examples are used for test_loader, and there is no examples for validation set. And the "best accu" reported in the paper is the max of all test accuracy calculated in all epochs ?
But some papers reserve 5000 clean examples as a validation set, and only use 45000 examples which are injected noise for training, and test accu is calculated in 10000 test examples. The "best accu" they reported is the test accuracy in the epoch when validation accu in clean 5000 examples is best, which may be low than the potential best test accu.
Hi! As you are commenting and it is noted in the paper, we only use train and test sets as other works do. We report best and last to show that our approach exhibits little overfitting to noisy labels, as oposed to many related work, where at the end of the training the accuracy has substantial drops. Our main reason for making this decision is that the existence of a clean set is not straightforward in a label noise scenario and simply doing a semi-supervised training with that clean "validation" set would lead to better performance.
Please, let us know if you have further doubts.
Okay. Very thanks for your reply.