How to train the best model
HaoWenJohn opened this issue · comments
Hello, I used your code to search for more than 200 models. During training, they have good accuracy of verification set (about 0.86), but in the subsequent calculation of pr-AUC, their performance (0.1-0.3) is far from the best model you provided, and PR curve and violin plots also look very bad. I want to know if there is any other work besides changing the search times to find the effect like the model you provided. I'm sorry for my poor English. I wonder if you can understand me
Good question! (and yes, I can understand you well)
I'm not sure if this will 100% work, but you can fix the hyper-parameters in the random search to match the best model I got. Give that a try. Change these in the code:
beta_value =params["beta_value"]
codings_size =params["codings_size"]
conv_layers =params["conv_layers"]
start_filter_no =params["start_filter_no"]
kernel_size_1 = params["kernel_size_1"]
dilations = params["dilations"]
earlystop_patience=params["earlystop_patience"]
The solution (fixing the hyperparameters) is a bit "hacky" -- I've since made it easier in my later work.
Also, a VAE is naturally not deterministic, so you can expect variability between models, even with the same hyperparameters between models.
Finally, try training a bunch more models. Eventually you should get one that is closer.
Hope this helps.