The order of the parameters is reversed. The two parameter (save_every, total_epochs)positions should be interchanged.

[tianph]

examples/distributed/ddp-tutorial-series/multigpu.py

Line 102 in 35eb814

 mp.spawn(main, args=(world_size, args.total_epochs, args.save_every, args.batch_size), nprocs=world_size) 

[tianph]

It took me hours to find the bug. This is an unforgettable experience.

[tianph]

And it also need to use sampler.set_epoch() method at the beginning of each epoch.

[Steven Liu]

cc: @suraj813

[Suraj Subramanian]

@RuleNHao thanks for catching this!

set_epoch() is an important suggestion to ensure random shuffling; thank you for bringing this up, I'll update the example to include this.