BaguaSys / bagua

Bagua Speeds up PyTorch

https://tutorials-8ro.pages.dev/

improve low prec decentralized performance

wangraying opened this issue 3 years ago · comments

Rui Wang commented 3 years ago

try:

tune bucket size
make computation and communication overlap
... .etc.