[Q] Initialize multiple runs and update them simultaneously
ksmdnl opened this issue · comments
Thank you for the never ending support of the API!
I have a scenario where I have multiple forward passes in each step. I imagine initialising multiple wandb runs and update them independently. Is there a feature that might support this use case?
This is roughly what I would like to have:
wandb_run_1 = wandb.init(...)
wandb_run_2 = wandb.init(...)
runs = [wandb_run_1, wandb_run_2]
for _ in range(step_nums):
for pass_idx in range(num_forward_pass):
runs[pass_idx].log(...)
The closest I found is #1063, but this is not exactly the same scenario, since it runs each run one after the other.
I’ll appreciate any kind of hints!
hey @ksmdnl - currently we don't support logging more than 1 run in the same process. this would have to be done sequentially. alternatively, perhaps you can utilize Python's multiprocessing library to spin up separate processes. Each process can handle its own W&B run, allowing you to independently log different forward passes or experiments. Please let me know if you have further questions or would like to discuss further!
Hi @umakrishnaswamy, thanks for the input!
If I understood it correctly, I would then need to initialize the API object in as many processes as the number of forward pass and log the corresponding data there. However, when each process finished, so as each corresponding run is also finished. Thus, each step the same thing should be started again. But I'm not sure if this will help because I'd like to keep each run (without initializing them again every step) and update them one step at a time. Is my understanding correct or do you imagine another use case/approach using multiprocessing?
To be clear, in my scenario it's only one experiment with multiple forward passes per step (which includes the optimization) and I would like to log the output of each pass.
@ksmdnl - got it, unfortunately we do not support this right now since we can only update one run at a time, and you would have to terminate the other run in order to log to the new run. I'd be happy to make a feature request for this, and I'll update this thread if any progress arises on it.
@umakrishnaswamy yes, that would be very helpful. Thanks!