[Features] support gradient checkpointing for memory saving
zguo0525 opened this issue · comments
Hello! What do you mean by gradient checkpointing? Like the checkpoint method from torch?
huggingface / nanotron
Minimalistic large language model 3D-parallelism training