Season-invariant GNSS-denied visual localization for UAVs

This repository contains data and code for manuscript J. Kinnari, F. Verdoja, V. Kyrki "Season-invariant GNSS-denied visual localization for UAVs".

Setting up

Clone this repository on your computer.

Download the checkpoint file exp45_clean.pt from here and place it in folder checkpoints.

Install Anaconda. Create a new environment using the provided yml file:

conda env create -f environment.yml

Remark: The environment.yml file currently contains more packages than necessary.

Simple example

A simple example for computing similarity between two 96 by 96 pixel RGB images is provided in computeSimilarity.py. A few sample images can be found in folder sample_images. To check operation of the trained model with pairs of images, run

python3 computeSimilarity.py ../sample_images/sample1_anc.png ../sample_images/sample1_pos.png

Configuration

The file configs/configuration.yml contains various configuration details with comments for each parameter.

Training

After downloading the data to folder data as instructed at the end of this file, you may train the network yourself using the script trainSimilarityModel.py.

Citing

If this code is useful in your research, please consider citing:

@ARTICLE{9830867,
  author={Kinnari, Jouko and Verdoja, Francesco and Kyrki, Ville},
  journal={IEEE Robotics and Automation Letters}, 
  title={Season-Invariant GNSS-Denied Visual Localization for UAVs}, 
  year={2022},
  volume={7},
  number={4},
  pages={10232-10239},
  doi={10.1109/LRA.2022.3191038}}

Downloading data

What does one dataset contain?

Each dataset contains a number of aligned images, acquired at different times, from the same area, at a resolution of approximately 1 m/pixel.

Where to store images

Place each image in the corresponding folder under folder data or adjust paths in configuration.yml to match your configuration.

Acquiring Woodbridge and Fountainhead datasets

Rows 1 and 2 in the table below are from this paper by Goforth and Lucey.

Acquiring ge_area datasets

Rows 3 through 18 in the table below are acquired using Google Earth Pro desktop. Google Earth license does not allow distribution of data acquired with Google Earth Pro and therefore the dataset cannot be distributed as images directly. Also, Google Earth Pro does not appear to have a suitable API for exporting these kinds of images using a script.

However, in order to acquire the same dataset, you can download the same images with a bit of manual work according to these instructions:

Import each placemark file (.kmz) found in folder ge_placemarks into Google Earth Pro
Click View/Historical Imagery to enable historical imagery.
From Layers panel, remove tickmarks next to each layer selection.

For each placemark:

Double-click the placemark to center the view in the correct position, at the correct altitude
From Places tab, remove the tickmark for that placemark in order to remove it from view.
Slide the slider to one of the month/date combinations listed in table below.
Click the "Save Image" button. From Map Options, disable the items "Title and Description", "Legend", "Scale", "Compass" and "HTML Area"
From "Resolution", select "Maximum (4800 x 2987)"
Click "Save Image". Save the image with filename "<year>_<month>_<number>.jpg" in the folder defined in the table below.

These instructions were written for Google Earth Pro 7.3.4.8248 (64-bit) on Windows.

Listing of datasets

Row	Dataset name	Imaging months	Dataset use	Path
1	woodbridge	2006/8 2008/8 2010/8 2013/8 2015/8 2017/8	tra	../data/USGS/sat_data/woodbridge/images
2	fountainhead	2009/6 2012/5 2016/7	tra	../data/USGS/sat_data/fountainhead/images
3	ge_area1	2011/3,4,6 2015/8,8 2018/5	tra	../data/google_earth_exports/area1
4	ge_area2	2011/3,4,6 2013/8 2014/7 2015/8 2017/8 2018/5	tra	../data/google_earth_exports/area2
5	ge_area4	2013/8 2014/7 2015/8 2017/7 2018/5,5 2020/3,3	tra	../data/google_earth_exports/area4
6	ge_area5	2011/3 2013/3,8 2014/7 2015/8 2017/7,8 2018/5 2020/6	tra	../data/google_earth_exports/area5
7	ge_area6	2002/7 2011/3 2013/3,8 2014/7 2015/8 2017/7 2018/5 2020/6	tra	../data/google_earth_exports/area6
8	ge_area12	2013/4 2014/7 2017/7 2018/5 2020/6	tra	../data/google_earth_exports/area12
9	ge_area13	2011/3,9 2012/5 2013/4 2020/3,6	tra	../data/google_earth_exports/area13
10	ge_area14	2011/6 2012/5 2013/4,8 2015/8	tra	../data/google_earth_exports/area14
11	ge_area15	2015/8 2016/7 2018/5 2020/4	tra	../data/google_earth_exports/area15
12	ge_area3	2006/5 2008/4 2011/9 2012/5 2013/4,8 2014/5,7 2015/8 2017/7 2018/5,5 2020/3,3,6	tes	../data/google_earth_exports/area3
13	ge_area3	2006/5 2008/4 2011/9 2012/5 2013/4,8 2014/5,7 2015/8 2017/7 2018/5,5 2020/3,3,6	mcl	../data/google_earth_exports/area3
14	ge_area7	2002/7 2011/3 2013/3,8 2014/7 2015/8 2017/7 2018/5 2020/6	mcl	../data/google_earth_exports/area7
15	ge_area8	2008/5 2010/7 2011/8 2013/4 2014/4 2017/5,8 2019/8 2020/3,6	lik	../data/google_earth_exports/area8
16	ge_area9	2011/3,6 2017/8 2019/7 2021/7	lik	../data/google_earth_exports/area9
17	ge_area10	2012/5 2014/8 2020/6 2021/4	lik	../data/google_earth_exports/area10
18	ge_area11	2005/2 2013/6 2014/8 2019/4 2020/6 2021/4	lik	../data/google_earth_exports/area11

"Dataset use" column specifies the use of this dataset:

tra: used in training the network
tes: used in testing the network
mcl: used for Monte Carlo localization tests
lik: used for estimating likelihood from similarity score

aalto-intelligent-robotics / sivl