wanchaol / spmd

spmd layer implementation

Run distributed tensor tests:

Need 4 GPU devices to run:

pytest tests/tensor/test_tensor.py

pytest tests/tensor/test_ddp.py