one shot CNN for both detection and feature extraction
Grabber opened this issue · comments
Is there any code showing the loss function to train the current residual network?
What the authors think about having a single network to do detection and feature extraction? Is it possible?
The training code is not yet published, but that should be a follow-up at some point.
About detection and feature extraction in a single network: We did some initial experiments where we fine-tuned the final layer of a pre-trained py-faster-rcnn on VOC, but it did not work that well for re-identification / tracking. It would be interesting if you get reasonable results, though.
Are you using a cosine loss function on the network that is extracting features for re-identification? Could you share it, please?
What I'm thinking about is use object detection patch-features as the input for re-identification network, on a single shot. The object detection features already know how to describe pedestrians, for example. I think it is a better idea than just feeding image pedestrian-patches to a new network.
What do you think?
Sorry for the late reply. We do use a cosine loss function, but haven't found the time to release the training code yet. I will let you know when we get there. Until then, you could experiment with some other well established loss formulations (e.g., [1,2]) and check how well they perform when providing as input the faster-rcnn features. The tracker supports changing from cosine to Euclidean metric, just change "cosine" to "euclidean" in deep_sort_app.py:162.
[1] https://arxiv.org/abs/1703.07737
[2] https://arxiv.org/abs/1511.05939
Closing this one down due to inactivity.
@nwojke Is the code for training on objects other than pedestrians out by now?
Is the code for training on objects other than pedestrians out by now?
Please see @nwojke 's other repo cosine_metric_learning for training code and associated reference.