parameter initialization/sensitivity (question)

[jdvylder]

Hi
I just started exploring with your toolbox and it looks very useful. I'm just curious about the parameters used to create the Latin Hyper cube, in one example it has a size 6 in another one it uses 50 samples. Based on the paper, I assume this hypercube generates an initialization of the model of your metric space based on the input hyper parameters, before the actual optimization (correct me if I'm wrong on this). I'm curious if there is a rule of thumb for correctly setting this value, and how sensitive your optimizer is to an underestimate of this value.

Thanks

Jonas

[icouckuy]

I assume you want to use GPFlowOpt for the hyperparameter optimization of your (deep learning) model. Then, yes, a latin hypercybe of 6 means that 6 hyperparameters are chosen to build the initial GP model.

There is absolutely no way to know what the ideal size is. If I remember correctly, (dimension*10)+1 was once proposed as a rule of thumb. The lower the better (as you need less expensive training cycles of your DL model), but not too low as the optimization might get stuck due to problems in the GP model building. For very low sizes (like 5 initial points) it is best to enable MCMC of the hyperparameters of the GP model to get a better variance estimate, but this comes at the cost of speed.

[jdvylder]

thanks