Modified Andrej's and William's awesome code to provide a benchmark script and show how to train a model from scratch at scale using Lightning.
- DeepSpeed Stage 3 Example
- FairScale Fully Sharded Example
- Benchmark Suite (Fairscale and DeepSpeed capabilities) + Streamlit Visualization
MIT