Evaluation time on transfer learning

Question

Evaluation time on transfer learning

bbln33 opened this issue a year ago · comments

Dear author,

Thanks for nice works!
I have a question of the inference time on transfer learning.
As you described in README, I run the evaluate script below:

python3 ./tools/train_net.py \
--eval-only  \
--num-gpus 8 \
--config-file ./configs/COCO-InstanceSegmentation/CLIP_fast_rcnn_R_50_C4_ovd.yaml \
MODEL.WEIGHTS ./pretrained_ckpt/regionclip/regionclip_finetuned-coco_rn50.pth \
MODEL.CLIP.OFFLINE_RPN_CONFIG ./configs/COCO-InstanceSegmentation/mask_rcnn_R_50_C4_1x_ovd_FSD.yaml \
MODEL.CLIP.BB_RPN_WEIGHTS ./pretrained_ckpt/rpn/rpn_coco_48.pth \
MODEL.CLIP.TEXT_EMB_PATH ./pretrained_ckpt/concept_emb/coco_48_base_cls_emb.pth \
MODEL.CLIP.OPENSET_TEST_TEXT_EMB_PATH ./pretrained_ckpt/concept_emb/coco_65_cls_emb.pth \
MODEL.ROI_HEADS.SOFT_NMS_ENABLED True \

I use NVIDIA 3090 8 gpus, but I guess it is little bit too long for evaluation.
Is is possible to share the inference time on transfer learning?

Haochen Li · Answer 1 · Wed Apr 12 2023 18:28:55 GMT+0800 (China Standard Time)

Hi！
Thanks for this great work！
However，I face the same thing. When I inference on a V100, it takes 10 seconds to infer 1 image. Is that normal?

Haochen Li · Answer 2 · Wed Apr 12 2023 20:59:15 GMT+0800 (China Standard Time)

OKAY I have found my mistake, bad weight loading. FYI, mine is 1s/ iter.

Shiyu Zhao · Answer 3 · Thu Apr 20 2023 04:48:46 GMT+0800 (China Standard Time)

Hey @bbln33, I met the same problem. It takes me ~1 hour on COCO validation, but a standard mask-rcnn provided by detectron2 only takes minutes. Did you solve it? Could you provide us how long it takes with your environment?

bbln33 · Answer 4 · Mon Apr 24 2023 08:07:20 GMT+0800 (China Standard Time)

In my case, the whole inference time is around 4 hours. I found that it depends on the MODEL.ROI_HEADS.SOFT_NMS_ENABLED option. When it sets true, it takes more time than false.