wandb number of random search iterations - default value

[lumliolum]

Hello,

While creating sweep for a model, hyperparameter method in sweep_config is either random or grid. The code for this is given here (see line 41)

tabular-benchmark/launch_config/utils.py

Lines 36 to 51 in 3ea416e

  
 sweep_config = { 
     "program": "run_experiment.py", 
     "name": name, 
     "project": project, 
     "method": "grid" if default else "random", 
     "metric": { 
         "name": "mean_test_score", 
         "goal": "minimize"  # RMSE 
     } if regression else { 
         "name": "mean_test_score", 
         "goal": "maximize"  # accuracy 
     }, 
     "parameters": dict(model_config, **data_transform_config) 
 } 
  

If the method is random, then what is number of random iterations that wandb will use ?

I checked the wandb documentation here about sweep strategy : https://docs.wandb.ai/guides/sweeps/define-sweep-configuration#configuration-keys

but couldn't find the default value of random search iterations

[LeoGrin]

If the method is random, wandb will keep on going until you tell it to stop. In the updated version of the code, I shared some code (https://github.com/LeoGrin/tabular-benchmark/blob/refactoring/src/launch_benchmarks/launch_on_cluster.py) to automatically stop wandb sweeps when enough runs have been finished.