Multi-node training
EphChem opened this issue · comments
Hi there,
Thank you so much for this release!
When trying to run multi-node training, I can see that this repo is equipped to do this, when I see the following lines:
Line 57 in da25b61
Line 42 in da25b61
Have you trained using multiple nodes (not just multiple GPUs) where you have to provide 2 different ip addresses from within the docker containers you provided in this repo? And has this worked for you? When I execute training on two different machines, the code hangs and I dont see any terminal printouts...
Thank you in advance!