microhhh / duality-diagram-similarity

Efficient solution to the initialization selection of DNNs in transfer learning using duality diagrams as a similarity measure framework.

Duality Diagram Similarity: a generic framework for initialization selection in task transfer learning
Kshitij Dwivedi, Jiahui Huang, Radoslaw Martin Cichy, Gemma Roig
ECCV 2020

Here we provide the code to replicate our results on Taskonomy and Pascal VOC transfer benchmark. You can also find the implementation of our old method published in CVPR 2019 based on RSA in these links paper, code

• To assess similarity between two tasks, we extract the features of the Deep Neural Networks(DNNs) trained on these tasks

• We then create the Duality Diagram of a task from extracted feature matrix.

• We finally compare the Duality Diagrams of two tasks to assess their similarity.

Setup

• Code uses standard python libraries numpy, scipy, scikit-learn, pandas so it should run without installing additional libraries
• Download saved features of Taskonomy and Pascal VOC models from this link , and save the features in ./features directory.
• Download taskonomy groundtruth transfer learning results for affinities and winrate and save them in ./affinities folder

Taskonomy

• Run python computeDDS_taskonomy.py to compute DDS between Taskonomy models
• Compare the DDS with transfer learning performance by running the jupyter notebook : DDS_vs_transferlearning(Taskonomy).ipynb
• The comparison results of DDS with transfer learning using Taskonomy images should be identical to Table below

Pascal VOC

• Run python computeDDS_pascal.py to compute DDS between Taskonomy models and Pascal VOC model
• Compare the DDS with transfer learning performance by running the jupyter notebook : DDS_vs_transferlearning(Pascal).ipynb
• The comparison results of DDS with transfer learning using Pascal images should be identical to Table below

Cite

If you use our code please consider citing the paper below

@inproceedings{dwivedi2020DDS,
  title={Duality Diagram Similarity: a generic framework for initialization
               selection in task transfer learning},
  author={Kshitij Dwivedi and
               Jiahui Huang and
               Radoslaw Martin Cichy and
               Gemma Roig},
  booktitle={Proceedings of the European Conference on Computer Vision (ECCV)},
  year={2020}
}