How much (continous) variables can the optimizer handle in practice

[Hannes Fassold]

Is there an estimation how many variables the optimizer can handle in practice ?

I am especially interested in floating-point variables with a value range [0, 1].

For my application, I would need the optimizer to be able to support a few hundreds of such variables.
Do you think the optimizer can handle that ?

[xingyousong]

Hi @hfassold, for the case of hundreds of continuous variables, global search methods such as Bayesian Optimization (which OSS Vizier is defaulted to) may not be appropriate (for various reasons, especially due to Gaussian Process scaling).

Instead, local search methods (usually via evolutionary algorithms) are much better. You can try the following:

1. Use CMA-ES, via study_config.algorithm = 'CMA_ES' which is a classic method for continuous search spaces.
2. Use any evolutionary method (Regularized Evolution, Hill-Climbing, etc.) via our PyGlove integration instead: https://oss-vizier.readthedocs.io/en/latest/advanced_topics/pyglove/vizier_as_backend.html

[Hannes Fassold]

Great, thank you ! I heard about CMA-ES, it is supposed to be a great black-box (gradient-free) optimization algorithm.