Giters

microsoft / RegionCLIP

[CVPR 2022] Official code for "RegionCLIP: Region-based Language-Image Pretraining"

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

The result of RPN is close to 0 for zero-shot inference of own dataset, however, the result of GT is very good. What should I do?

QHCV opened this issue · comments

commented

Dear Author,
Very great work. I am getting close to 0 results for inference at RPN at zero-shot inference and very good results for inference at GT. What should I do in this case and the results are poor when doing Transfer Learning, I suspect that it is also due to RPN. What should I do?
think you.

commented

What I do is to merge the weights of RPN and pretrained models and don't provide RPN checkpoint, i.e., only provide one model checkpoint with MODEL.WEIGHTS. Then the mAP in paper can be reproduced.

commented

Thanks for your interest in our work. According to your descriptions, the low performance is caused by localization errors. You could try training an RPN on your own dataset. This should provide a reasonable performance between 0 and GT boxes.

commented

g

I have also encountered the same situation. Would you be willing to further explain this phenomenon? I also believe that the decrease in accuracy is due to a localization issue, but I do not understand why the AP value would be close to 0.

commented

Thanks for your interest in our work. According to your descriptions, the low performance is caused by localization errors. You could try training an RPN on your own dataset. This should provide a reasonable performance between 0 and GT boxes.

Hello! I have tried to train an RPN on my own dataset. However, even if the RPN model shows that it could predict the catogories on the AP around 60, when I try to use the RPN model weight in RegionCLIP (I just simplily replace the model weight, I guess it may have some mistakes but I didn't find), the AP is not closed to 0 but relatively low (around 5 on some categories and also be closed to 0 on the others). And the GT boxes performs quite good. Could you please give me some advice concerning using my own RPN model?

commented

@whhong5 According to your descriptions, the RPN you loaded from RegionCLIP seems a bit off. If your own RPN is using exactly the same module as RegionCLIP's RPN, please check the weight loading. If not, you might need to additionally check the module definition & preprocess & feedforward functions.