This repository is a Pytorch implementation for the Detector-oblivious part of the paper.
Xuelun Shen, Cheng Wang, Xin Li, Qian Hu, Jingyi Zhang. "A Detector-oblivious Multi-arm Network for Keypoint Matching."
Based on our findings, we could use SuperGlue with any keypoint detector without a time-consuming re-training process.
This code is based on Python (3.8.8) and Pytorch (py3.8_cuda11.1_cudnn8.0.5_0), and tested on Ubuntu 18.04 with GeForce RTX 3090.
You could also use other environments and GPU, as long as the code can run successfully, and should be able to get similar results as in the paper.
You could install necessary packages by the command below:
pip install numpy opencv-python matplotlib imagesize tqdm h5py imageioDownload datasets in the compressed file data-DETO.zip from Google drive, unzip it and rename the folder from data-DETO to data.
Download pretrained model in the compressed file weights-DETO.zip from Google drive, unzip it and rename the folder from weights-DETO to weights.
Put the data folder into the source code folder.
Put the weights folder into the models folder.
If we name the source code folder as DETO.
The directory structure should be like this:
[ 🖱️ Click to show the file structure]
DETO
│
└───assets
│
└───data
│ │
│ └───MegaDepth
│ │
│ └───scannet
│ │
│ └───sun3d
│ │
│ └───yfcc
│
└───models
│
└───weights
│
└───r2d2_WASF_256d.pt
│
└───r2d2_WASF_N16.pt
│
└───superglue_indoor.pth
│
└───superglue_outdoor.pth
│
└───superpoint_v1.pthWe use branch in git to manage different experimental settings, and name the branch using the p-d-c format that are from the table below.
[ 🖱️ Click to show the result tables]
For example, if we switch the branch from main to SP-SP-SP, and then run the code under this branch (SP-SP-SP), we will get the #1 experimental result in the table.
Therefore, in addition to branch main, this repository also contains 12 branches, corresponding to the 12 experiment numbers in the table above.
By comparing the source code of different branches, we can easily observe which changes have affected performance.
[ 🖱️ Click to switch the branch]
| Experiments | p | d | c |
|---|---|---|---|
| 👉main | -- | -- | -- |
| #1 | SP | SP | SP |
| #2 | R2D2 | SP | SP |
| #3 | SP | R2D2 | SP |
| #4 | SP | SP | R2D2 |
| #5 | SP | SP | RAND |
| #6 | SP | SP | ZERO |
| #7 | SP | SP | ONE |
| #8 | SIFT | SIFT | SIFT |
| #9 | SIFT | SP | SIFT |
| #10 | SIFT | SP | RAND |
| #11 | SIFT | SP | ZERO |
| #12 | SIFT | SP | ONE |
🥇 This source code is heavily borrowed from SuperGlue. If you use the code in your research, please cite the SuperGlue and follow its license.
🥇 The R2D2 code is from R2D2. If you use the code in your research, please cite the R2D2 and follow its license.
🥇 Data MegaDepth is from D2-Net and MegaDepth.
🥇 Data yfcc is from OA-Net and YFCC100m.
🥇 Data scannet is from SuperGlue and ScanNet.
🥇 Data sun3d is from OA-Net and SUN3D.
- Update code of experiments from #1 to #12.
- Add a Chinese-Simplified README.