Detection of Marine Debris with Sentinel-2

Rußwurm M, Venkatesa J. S., Tuia D. (2023; in submission) Large-scale Detection of Marine Debris in Coastal Areas with Sentinel-2

Getting Started

We provide a python package for easy installation and model predictions

Installation of the Python Package

pip install marinedebrisdetector

# alternative from GitHub directly
pip install git+https://github.com/marccoru/marinedebrisdetector

You now have a marinedebrisdetector program to make predictions of Sentinel-2 tif images on the console. Check marindebrisdetector --help for information on parameters and options.

Predict the qualitative test images from the paper with this command.

marinedebrisdetector --plot-qualitative

predict a Sentinel-2 scene (Durban) with these commands

# download example scene (~300 MB) to ("durban_20190424.tif")
marinedebrisdetector --download-durban

# predict the durban example scene
marinedebrisdetector durban_20190424.tif

the prediction should be in the same folder named durban_20190424_prediction.tif. You can inspect the prediction scene in any image viewer, but best in a GIS program QGIS or ArcGIS that allows you inspect and overlay other georeferenced data.

The tested prediction times of the scene prediction above are:

11 sec on NVIDIA GeForce RTX 3090
15 sec on NVIDIA GeForce GTX 970
11 minutes on Macbook Pro CPU

Pre-trained Models

We provide pre-trained weights for 12-channel Sentinel-2 imagery. A detailed list of weights can be found on the models page

pretrained segmentation models can be loaded via the torch hub in python

import torch

torch.hub.load("marccoru/marinedebrisdetector", "unetpp")
torch.hub.load("marccoru/marinedebrisdetector", "unet")

# trained without label refinement (can lead to thinner more fine-grained predictions)
torch.hub.load("marccoru/marinedebrisdetector", "unetpp", label_refinement=False)

The marinedebrisdetector defaults to the unetpp/unet++.

Datasets

We trained the underlying Unet++ segmentation model (implementation from Pytorch Segmentation Models) on an aggregated a combination of existing datasets for training FloatingObjects Dataset (Mifdal et al., 2020), Marine Debris Archive (Kikaki et al., 2022), S2Ships (Ciocarlan et al., 2021), and newly annotated a refinedFloatingObjects dataset and Sentinel-2 images of the Plastic Litter Projects (Papageorgiou et al., 2022; under review)

More details in a dedicated data page. Executing the training script (see this "model training" page) with --download will automatically download and uncompress the required datasets (116 GB (uncompressed)).

Model Training

We provide a a training script powered by Pytorch Lightning and Weights and Biases to train new models and reproduce our results. More details on training commands and ablations in a dedicated page for model training

Random Forest Comparisons

We compare to a random forest trained on 1) the original MARIDA dataset (Kikaki et al., 2022) and 2) a random forest trained on our dataset. You should be able to reproduce the results in these scripts. The required data and random forest classifier is downloaded automatically.

python comparisons/test_rf_original_data.py
python comparisons/test_rf_our_data.py

MarcCoru / marinedebrisdetector