Problems with DDP + Optuna

[Yunchong Gan]

When I try to search with optuna, on 8-gpus with ddp strategy training.
The sweeper will start 8 groups of different hyperparameters, so the params shape doesn't match on each gpu.