Draw a roc metric plot from another dataset
DaehanKim-Korea opened this issue · comments
I tried drawing it using "--load snapshot.pt", but it is lower than the performance confirmed during training.
Assuming you have a validation set of the same type other than the dataset used for training and testing. In this case, how can I draw the roc curve using snapshot.pt?
Can you tell me the right way?
At first glance, this seems to be the right way. Train on your initial dataset A (with train split A1 and test split A2). Load the snapshot, change to dataset B, and let the runner validate on B.
Did you try swapping the validation sets? So training on A1 with the "test split" being B, and then loading the snapshot, changing to A2, and letting the runner validate on A2?
The process you mentioned seems to include training.
I want to plot on a test dataset without training.
I've made some modifications, but I'm still testing on the training data... What am I doing wrong?
Well, first of all, since the snapshot remembers the trained epochs, loading it and continuing with another dataset should not retrain it unless you increase the number of epochs. In either case, you can just set the number of epochs to zero to make double sure (here).
The ROC plots should always report the test AUC.