ggsonic / CosPlace

Official code for CVPR 2022 paper "Rethinking Visual Geo-localization for Large-Scale Applications"

Rethinking Visual Geo-localization for Large-Scale Applications

This is the official repository for the CVPR 2022 paper Rethinking Visual Geo-localization for Large-Scale Applications. The paper presents a new dataset called San Francisco eXtra Large (SF-XL, go here to download it), and a highly scalable training method (called CosPlace), which allows to reach SOTA results with compact descriptors.

The images below represent respectively:

1. the map of San Francisco eXtra Large
2. a visualization of how CosPlace Groups (read datasets) are formed
3. results with CosPlace vs other methods on Pitts250k (CosPlace trained on SF-XL, others on Pitts30k)

Train

After downloading the SF-XL dataset, simply run

$ python3 train.py --dataset_folder path/to/sf-xl/processed

the script automatically splits SF-XL in CosPlace Groups, and saves the resulting object in the folder cache. By default training is performed with a ResNet-18 with descriptors dimensionality 512 is used, which fits in less than 4GB of VRAM.

To change the backbone or the output descriptors dimensionality simply run

$ python3 train.py --dataset_folder path/to/sf-xl/processed --backbone resnet50 --fc_output_dim 128

You can also speed up your training with Automatic Mixed Precision (note that all results/statistics from the paper did not use AMP)

$ python3 train.py --dataset_folder path/to/sf-xl/processed --use_amp16

Run $ python3 train.py -h to have a look at all the hyperparameters that you can change. You will find all hyperparameters mentioned in the paper.

Dataset size and lightweight version

The SF-XL dataset is about 1 TB. For training only a subset of the images is used, and you can use this subset for training, which is only 360 GB. If this is still too heavy for you (e.g. if you're using Colab), but you would like to run CosPlace, we also created a small version of SF-XL, which is only 5 GB. Obviously, using the small version will lead to lower results, and it should be used only for debugging / exploration purposes. More information on the dataset and lightweight version are on the README that you can find on the dataset download page (go here to find it).

Reproducibility

Results from the paper are fully reproducible, and we followed deep learning's best practices (average over multiple runs for the main results, validation/early stopping and hyperparameter search on the val set). If you are a researcher comparing your work against ours, please make sure to follow these best practices and avoid picking the best model on the test set.

Limitations

Given that we use only 8 out of 50 groups, the code in datasets/train_dataset.py only allows for groups with L=0 to be used for training, to keep the code more readable. Therefore you can use up to NxNx1 groups, which is up to 25 when N=5 as in the paper.

Test

You can test a trained model as such

$ python3 eval.py --dataset_folder path/to/sf-xl/processed --backbone resnet50 --fc_output_dim 128 --resume_model path/to/best_model.pth

You can download plenty of trained models below.

Model Zoo

In the table below are links to models with different backbones and dimensionality of descriptors, trained on SF-XL. If you want to use these weights in your own code, make sure that the model is the same as ours: CNN backbone -> L2 -> GeM -> FC -> L2.

Model	Dimension of Descriptors
	32	64	128	256	512	1024	2048
ResNet-18	link	link	link	link	link	-	-
ResNet-50	link	link	link	link	link	link	link
ResNet-101	link	link	link	link	link	link	link
ResNet-152	link	link	link	link	link	link	link
VGG-16	-	link	link	link	link	-	-

Or you can download all models at once at this link

Issues

If you questions regarding our code or dataset, feel free to open an issue or send an email to [e-mail]

Acknowledgements

Parts of this repo are inspired by the following repositories:

• CosFace implementation in PyTorch
• CNN Image Retrieval in PyTorch (for the GeM layer)
• Visual Geo-localization benchmark (for the evaluation / test code)

Cite

Here is the bibtex to cite our paper

@inProceedings{Berton_CVPR_2022_cosPlace,
  author = {Berton, Gabriele and Masone, Carlo and Caputo, Barbara},
  title = {Rethinking Visual Geo-localization for Large-Scale Applications}, 
  booktitle = {CVPR},
  month = {June}, 
  year = {2022}, }