Eval of coco_2017_val for `coco_65_cls_emb.pth'

[Priyank Pathak]

If I want to evaluate the performance on coco_2017_val (80 classes) for

WT=./pretrained_ckpt/regionclip/regionclip_pretrained-cc_rn50.pth
CLIP_WT=./pretrained_ckpt/concept_emb/coco_65_cls_emb.pth

Currently, I'm doing

MODEL_CONFIG=configs/COCO-InstanceSegmentation/CLIP_fast_rcnn_R_50_C4_ovd_coco80.yaml
WT=./pretrained_ckpt/regionclip/regionclip_pretrained-cc_rn50.pth
CLIP_WT=./pretrained_ckpt/concept_emb/coco_65_cls_emb.pth
CUDA_VISIBLE_DEVICES=0 python ./tools/train_net.py --eval-only  --num-gpus 1 \
    --config-file $MODEL_CONFIG MODEL.WEIGHTS $WT MODEL.CLIP.TEXT_EMB_PATH $CLIP_WT \
    MODEL.CLIP.CROP_REGION_TYPE GT MODEL.CLIP.MULTIPLY_RPN_SCORE False \
    MODEL.ROI_HEADS.NUM_CLASSES 65

# [10/11 19:44:37 d2.evaluation.testing]: copypaste: Task: bbox
# [10/11 19:44:37 d2.evaluation.testing]: copypaste: AP,AP50,AP75,APs,APm,APl
# [10/11 19:44:37 d2.evaluation.testing]: copypaste: 8.4155,8.4626,8.4114,6.4665,8.4770,9.6864

Is this allowed? or do I have to switch the RPN head to ./pretrained_ckpt/concept_emb/coco_80_cls_emb.pth

like this

WT=./pretrained_ckpt/regionclip/regionclip_pretrained-cc_rn50.pth
CLIP_WT=./pretrained_ckpt/concept_emb/coco_80_cls_emb.pth

CUDA_VISIBLE_DEVICES=0 python ./tools/train_net.py --eval-only  --num-gpus 1 \
    --config-file $MODEL_CONFIG MODEL.WEIGHTS $WT MODEL.CLIP.TEXT_EMB_PATH $CLIP_WT \
    MODEL.CLIP.CROP_REGION_TYPE GT MODEL.CLIP.MULTIPLY_RPN_SCORE False 

# [10/11 19:44:37 d2.evaluation.testing]: copypaste: Task: bbox
# [10/11 19:52:25 d2.evaluation.testing]: copypaste: AP,AP50,AP75,APs,APm,APl
# [10/11 19:52:25 d2.evaluation.testing]: copypaste: 58.1881,58.6332,58.1368,45.0736,65.9012,61.9111

Is this correct? Basically how to evaluate coco_65_cls_emb.pth on coco_2017_val (evaluation on basically 80 - 65 novel classes)