Test the GPU bandwidth of collectives operators like all-reduce, all-gather, broadcast and all-to-all primitives on single-node multi-GPU (2, 4, 8 cards) and multi-node multi-GPU (16 cards) setups, using only PyTorch and Python built-in packages.
Geek Repo:Geek Repo
Github PK Tool:Github PK Tool
