Visual Explanation for Deep Metric Learning
This work explores the visual explanation for deep metric learning and its applications. We show that the proposed framework can be directly deployed to a large range of metric learning applications and provides valuable information for understanding the model. See the arxiv paper for details.
We provide runnable demos for activation decomposition and Grad-CAM (with our variants) on Image Retrieval, Person Re-identification, Face Verification, and Geo-localization. If you find this repository useful in your project, please consider citing:
@article{zhu2021visual,
title={Visual explanation for deep metric learning},
author={Zhu, Sijie and Yang, Taojiannan and Chen, Chen},
journal={IEEE Transactions on Image Processing},
year={2021},
publisher={IEEE}
}
Requirement
- Python >= 3.5, Opencv, Numpy, Matplotlib
- PyTorch >= 1.0
- Tensorflow == 1.13.1 only for Geo-localization
Image Retrieval
Please download our pretrained model (Multi_similarity) and put it in "./Image_Retrieval/Model/". Then run the demo:
python Image_Retrieval/demo.py
You may download the CUB dataset to generate more results. Enjoy!
Person Re-identification
Please download our pretrained model (strong baseline) and put it in "./Person_Re-identification/Model/". Then run the demo:
python Person_Re-identification/demo.py
You may download the Market-1501 dataset for more results. Enjoy!
Face Verification
Please download our pretrained model (arcface) and put it in "./Face_Verification/Model/". Then run the demo:
python Face_Verification/demo.py
You may download the LFW or FIW dataset for more results. Enjoy!
Geo-localization
Please download our pretrained model package (Siamese-VGG) and put it in "./Geo-localization/Model/". Use tar -xvf model.ckpt.tar to extract. Then run the demo:
python Geo-localization/demo.py
Query pixel moving on the query street view image (left) The changing point-specific activation map on the retrieved aerial view image (right)
You may download the CVUSA dataset for more results. Enjoy! A cleaner yet stronger model is coming soon!
Reference
- https://github.com/bnu-wangxun/Deep_Metric
- https://github.com/lulujianjie/person-reid-tiny-baseline
- https://github.com/foamliu/InsightFace.git
- https://github.com/david-husx/crossview_localisation.git