Multi-GPU support is now experimental
xuezhongcailian opened this issue · comments
Feature
Does this repository currently support multi-GPU experiments
Motivation
Additional context
Hi, Not at the moment. For reference, there is a PR on that (#223) which requires some cleaning and a more robust / generic way to gather gpu info than using WORLD_SIZE directly.