Can I extract features of each detected ROI from yolo?

[CXZ116]

Hi:
I have been playing with yolo9000 and compare it with other object proposals such as selective search. The way I did in the previous projects was to get regions from the object proposal and feed the ROI images to AlexNet to get 4kb features per image ROI. Yolo basically calculate object regions and scores at one shot, and performs faster per image. I have not read the paper on detailed implementation yet, but would like to ask the questions first if quick answers can be found:

1. For each region and score, can I obtain the associated feature? Is there any available API to call or do I need to modify the code to make it happen?

2. I want to train new classes in addition to the available classes. My thought is to fix all the levels of weight except the last fully connected layer and fine tune only that layer in a hope that other levels of weights have been trained well enough. The reason to do that is I only need to train the new (relatively small) dataset and converge quickly. Unlike others I do not want to throw away trained classes, but add new classes to the available network. Would this approach work as a general case? Do I have to train full dataset (including yolo 9000 classes) at some time?

Thanks a bunch,

David

[Philippe Rémy]

@CXZ116 that's a very good question for which I don't have the answer.

This project does not contain the source code of Yolo.

In your case, you should ask directly to the guys here:
https://groups.google.com/forum/#!forum/darknet

Thanks!

[Vivek Loganathan]

@CXZ116
I want to do the same. Where you able to get the answer ?

[CXZ116]

Yes, I think I found solutions, Check: https://towardsdatascience.com/understanding-region-of-interest-part-2-roi-align-and-roi-warp-f795196fc193 For ROI pooling explanations. Another paper for your reference: “Spatial Pyramid Pooling in Deep Convolutional Networks for Visual Recognition” by He et al. Not sure if it helps you since what I need is a fixed length feature per ROI regardless of the object’s ROI size. Another solution is to add an autoencoder in parallel with the detection of ROI and class label, and use the annotation ROI images (MSE loss) to train the autoencoder. The feature embedding from the encoder is the feature output. David From: vivek87799 <[e-mail]> Sent: Monday, June 22, 2020 1:50 PM To: philipperemy/yolo-9000 <[e-mail]> Cc: David Zhang <[e-mail]>; Mention <[e-mail]> Subject: [EXTERNAL] Re: [philipperemy/yolo-9000] Can I extract features of each detected ROI from yolo? (#14) @CXZ116<https://urldefense.com/v3/__https:/github.com/CXZ116__;!!Nv3xtKNH_4uope0!xkkL_VTghWkU-yb5DzF1JtwciEHBnrqbze_5MkiIyolmxBOpQ3BrmYg9z3J9fhc$> I want to do the same. Where you able to get the answer ? — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub<https://urldefense.com/v3/__https:/github.com/philipperemy/yolo-9000/issues/14*issuecomment-647680247__;Iw!!Nv3xtKNH_4uope0!xkkL_VTghWkU-yb5DzF1JtwciEHBnrqbze_5MkiIyolmxBOpQ3BrmYg9EEFNGYs$>, or unsubscribe<https://urldefense.com/v3/__https:/github.com/notifications/unsubscribe-auth/ACXL6J3WGNOMVYBJQP5RO3DRX6KVZANCNFSM4ECWIMNQ__;!!Nv3xtKNH_4uope0!xkkL_VTghWkU-yb5DzF1JtwciEHBnrqbze_5MkiIyolmxBOpQ3BrmYg97Lp1qek$>.

[Vivek Loganathan]

Yes, I think I found solutions, Check: https://towardsdatascience.com/understanding-region-of-interest-part-2-roi-align-and-roi-warp-f795196fc193 For ROI pooling explanations. Another paper for your reference: “Spatial Pyramid Pooling in Deep Convolutional Networks for Visual Recognition” by He et al. Not sure if it helps you since what I need is a fixed length feature per ROI regardless of the object’s ROI size. Another solution is to add an autoencoder in parallel with the detection of ROI and class label, and use the annotation ROI images (MSE loss) to train the autoencoder. The feature embedding from the encoder is the feature output. David From: vivek87799 [e-mail] Sent: Monday, June 22, 2020 1:50 PM To: philipperemy/yolo-9000 [e-mail] Cc: David Zhang [e-mail]; Mention [e-mail] Subject: [EXTERNAL] Re: [philipperemy/yolo-9000] Can I extract features of each detected ROI from yolo? (#14) @CXZ116<https://urldefense.com/v3/https:/github.com/CXZ116;!!Nv3xtKNH_4uope0!xkkL_VTghWkU-yb5DzF1JtwciEHBnrqbze_5MkiIyolmxBOpQ3BrmYg9z3J9fhc$> I want to do the same. Where you able to get the answer ? — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub<https://urldefense.com/v3/https:/github.com/philipperemy/yolo-9000/issues/14*issuecomment-647680247;Iw!!Nv3xtKNH_4uope0!xkkL_VTghWkU-yb5DzF1JtwciEHBnrqbze_5MkiIyolmxBOpQ3BrmYg9EEFNGYs$>, or unsubscribe<https://urldefense.com/v3/https:/github.com/notifications/unsubscribe-auth/ACXL6J3WGNOMVYBJQP5RO3DRX6KVZANCNFSM4ECWIMNQ;!!Nv3xtKNH_4uope0!xkkL_VTghWkU-yb5DzF1JtwciEHBnrqbze_5MkiIyolmxBOpQ3BrmYg97Lp1qek$>.

Thank you very much for your reply.