How to best setup validation set

[Nkiruka Uzuegbunam]

I am using this repo to do facial recognition like insightface where model is trained on one set of data, and then used as a feature extractor for evaluation of the test set. But with training, I see my validation value is not going down. The val loss is static and does not go down, and the accuracy on the validation set is 0. The training accuracy, however, is high. (see image below). I am using --train_all so the train and validation should be included in the data for training, but yet val accuracy is 0.

I thought the reason for this bad training may be due to how I setup the splits. I setup my val such that my train and val have different IDs. Is this correct? I am running also with the --train_all flag and yet my val is 0?

Also, for my test set, I created my gallery to have a different set of ids from those in train and val. The queries all come from the ids in the gallery but uses images similar to train set. Is this the correct way to do this? I looked at prepare-*.py scripts and I can see that sometimes val is just a few samples from the train set, and has same ids in train set. What do you recommend for setting up a val set? Also, the model performs poorly on on the gallery and query set with Rank 1 at 3% so I'm not sure what I am doing incorrectly

[Tao]

我也遇到了这个问题 求郑大佬告知

[Zhedong Zheng]

Hi，@nikky4D @taozeze

Since we apply a classification model during training, it would be good to directly leave out some samples per class for validation. It is identical to the classification setting.

[Nkiruka Uzuegbunam]

Thanks. But if I'm training with train_all flag, this combines the ids in val and train. Shouldn't I see better accuracy then for the validation?