applying FIFO in Concurrent trials through Ray Tune

[Khalid ElHaj]

the max_concurrent feature is extremely helpful to speed up the optimization process though parallelization. However, the problem is that to start the next set of trials, it has to wait until ALL the concurrent trials (defined by max_concurrent) finishes. which is counter intuitive in my use case as some combination of hyperparameters takes too long to finish compared to other combinations. which leaves most of the CPUs Idle most of the time due to one trial taking too long. does HEBO need to have all concurrent running trials finish to suggest the next combination or is there a way to make it suggest the next combination once any of the concurrent running trails finish so no CPUs stays Idle until all of them finish?
btw, I'm using Ray Tune to deploy HEBO.