How to continue pre-training Bloom?

[ShinoharaHare]

Hi
I'm trying to continue pre-training the bloom-560m on my own dataset on a single GPU.
I modified this script to fit my case.
However, i cannot figure out how to load the checkpoint.

Is there any guide for what i'm doing?

[lwmlyy]

Hi,
did you find any solution on this?

[noob-ctrl]

@ShinoharaHare Hi, have you solved this problem？