This remote module of the Orfeo ToolBox provides a generic, multi purpose deep learning framework, targeting remote sensing images processing. It contains a set of new process objects that internally invoke Tensorflow, and a bunch of user-oriented applications to perform deep learning with real-world remote sensing images. Applications can be used to build OTB pipelines from Python or C++ APIs.
- Sample patches in remote sensing images with
PatchesExtraction, - Model training, supporting save/restore/import operations (a model can be trained from scratch or fine-tuned) with
TensorflowModelTrain, - Inference with support of OTB streaming mechanism with
TensorflowModelServe. The streaming mechanism means (1) no limitation with images sizes, (2) inference can be used as a "lego" in any OTB pipeline (using C++ or Python APIs) and preserving streaming, (3) MPI support available (use multiple processing unit to generate one single output image)
The otbtf module targets python developers that want to train their own model from python with TensorFlow or Keras.
It provides various classes for datasets and iterators to handle the patches images generated from the PatchesExtraction OTB application.
For instance, the otbtf.DatasetFromPatchesImages can be instantiated from a set of patches images
and delivering samples as tf.dataset that can be used in your favorite TensorFlow pipelines, or convert your patches into TFRecords.
The otbtf.TFRecords enables you train networks from TFRecords files, which is quite suited for
distributed training. Read more in the tutorial for keras.
tricks.py is here for backward compatibility with codes based on OTBTF 1.x and 2.x.
Below are some screen captures of deep learning applications performed at large scale with OTBTF.
- Landcover mapping (Spot-7 images --> Building map using semantic segmentation)
- Super resolution (Sentinel-2 images upsampled with the SR4RS software, which is based on OTBTF)
- Sentinel-2 reconstruction with Sentinel-1 VV/VH with the Decloud software, which is based on OTBTF
- Image to image translation (Spot-7 image --> Wikimedia Map using CGAN. So unnecessary but fun!)
For now you have two options: either use the existing docker image, or build everything from source.
Use the latest CPU or GPU-enabled image from dockerhub:
docker run mdl4eo/otbtf:3.3.3-cpu otbcli_PatchesExtraction -help
Read more in the docker use documentation.
Read more in the build from sources documentation.
- Reading the applications documentation will help, of course 😉
- A small tutorial on MDL4EO's blog
- in the
pythonfolder are provided some ready-to-use deep networks, with documentation and scientific references. - A book: Cresson, R. (2020). Deep Learning for Remote Sensing Images with Open Source Software. CRC Press. Use QGIS, OTB and Tensorflow to perform various kind of deep learning sorcery on remote sensing images (patch-based classification for landcover mapping, semantic segmentation of buildings, optical image restoration from joint SAR/Optical time series).
- Check our repository containing stuff (data and models) to begin with with!
- Finally, take a look in the
testfolder. You will find plenty of command lines for applications tests!
Every one can contribute to OTBTF. Just open a PR :)
@article{cresson2018framework,
title={A framework for remote sensing images processing using deep learning techniques},
author={Cresson, R{\'e}mi},
journal={IEEE Geoscience and Remote Sensing Letters},
volume={16},
number={1},
pages={25--29},
year={2018},
publisher={IEEE}
}