whuwuteng / Aerial_Stereo_Dataset

An evaluation of Deep Learning based stereo dense matching dataset shift from aerial images and a large scale stereo dataset

This Github will provide the detail information of our paper (JAG-D-23-00976) under review on International Journal of Applied Earth Observation and Geoinformation.

This is the Github repository for the stereo dense matching benchmark for AI4GEO project.

In order to discuss the transferability of deep learning methods on aerial dataset, we produce 6 aerial dataset covers 4 different area.

History

This work is an extension of our previous work, and the old version dataset is already published. In the ISPRS Conress 2022 in Nice, we presented an extension work as a poster, and the slide and the poster is provided.

Introduction

For stereo dense matching, there are many famous benchmark dataset in Robust Vision, for example, KITTI stereo and middlebury stereo. With the development of machine learning, especially deep learning, these methods usually need a lot of training data(or ground truth). For photogrammetry community, as far as we know, it is not easy to find these training data. We will publish our data as ground truth. The data is produced from original image and LiDAR dataset. To be noticed, the image and LiDAR should be well-registered.

Global information of dataset

For each dataset, the global information of the dataset is listed follow:

Dataset	Color	GSD(cm)	LiDAR($pt/m^2$)	Origin orientation	ICP refined	Outlier remove
ISPRS-Vaihingen	IR-R-G	8	6.7	✔	x	x
EuroSDR-Vaihingen	R-G-B	20	6.7	✔	x	x
Toulouse-UMBRA	R-G-B	12.5	2-4	x	☑	☑
Toulouse-Métropole	R-G-B	5	8	✔	x	x
Enschede	R-G-B	10	10	x	☑	☑
DublinCity	R-G-B	3.4	250-348	x	☑	x

In the table, the origin orientation accuracy influence the data accuracy, in order to improve the quality of the dataset, an ICP based Image-LiDAR is proposed to refine the orientation.

Dataset structure

The training and evaluation dataset is also provided, the structure of the folder is same with the old version.

Because the whole dataset is too large, so only the used in the paper is uploaded.

ISPRS-Vaihingen

All the training and testing data can be found on Google Drive, this is a newer version compare to the old version, the using origin image and LiDAR is same with old version.

Method

In the paper, we evaluate the state of the art methods of deep learning on stereo dense matching before 2020.

Pretrained models

The pretrained models are important in the paper, so we will also share the pretrained models and training setting in the paper.

TODO

• Image-LiDAR process
• Publish dataset V1 (use in the paper)
• Publish pretrained models
• Publish full dataset
• Publish the long paper on Arxiv

Stereo-LiDAR fusion

Based on the data generation, we also generate the Toulouse2020 data from IGN, and this data can be found in our CVPR photogrammetry and computer vision workshop paper. The Github site can be found here.

Citation

If you think you have any problem, contact [Teng Wu][e-mail]