DDP-compatible training infrastructure
BDannowitzHudl opened this issue · comments
@WongKinYiu mentioned that help is requested in building out DDP-compatible training infrastructure. Let's start that discussion here.
I suppose a better idea would be to start simple and then make it DDP
Hello, I wanted to ask if using Lightning https://lightning.ai/docs/pytorch/stable/advanced/ddp_optimizations.html adds any benefits?