How to resume training？
Somethinghx opened this issue · comments
python3 ./tools/train_net.py \
--num-gpus 1 \
--config-file ./configs/COCO-InstanceSegmentation/CLIP_fast_rcnn_R_50_C4_ovd.yaml \
MODEL.WEIGHTS ./pretrained_ckpt/regionclip/regionclip_pretrained-cc_rn50.pth \
MODEL.CLIP.OFFLINE_RPN_CONFIG ./configs/COCO-InstanceSegmentation/mask_rcnn_R_50_C4_1x_ovd_FSD.yaml \
MODEL.CLIP.BB_RPN_WEIGHTS ./pretrained_ckpt/rpn/rpn_coco_48.pth \
MODEL.CLIP.TEXT_EMB_PATH ./pretrained_ckpt/concept_emb/coco_48_base_cls_emb.pth \
MODEL.CLIP.OPENSET_TEST_TEXT_EMB_PATH ./pretrained_ckpt/concept_emb/coco_65_cls_emb.pth \
This is the command for training however it's not for resume.How should I resume my training from the checkpoint of the ‘instances_predictions.pth’ now that it didn't work by adding this？
@Somethinghx ‘instances_predictions.pth’ is the prediction results, instead of the checkpoint of training. Usually, the training will be resumed automatically, as long as your output folder has checkpoint file. Alternatively, a simple way to resume training is simply changing MODEL.WEIGHTS to the checkpoint file you had.
@YiwuZhong I am currently experimenting with the regionclip model,as such if i have any questions related to resume training ,can I ask it in this comment or shall i open another issue?