What this Repo is about

Semantic Segmentation is done by assigning each pixel in the image a class. This is a key problem in th field of computer vision. In this repo, we would look at using UNet to do this:

How to set up

what you need

Anaconda (download from here)

steps

Clone the Repo
open a terminal (bash)

In bash type the following

cd UNet
conda env create -f=environment.yml # this would download the necessary python packages from anaconda cloud
conda activate my_unet
cd data
tar -xvzf iccv09Data.tar.gz
mv iccv09Data raw # renames folder to raw for data integrity
cd ..
python src/processing.py
python src/train_unet.py

To Launch Jupyter notebook

type the following in your bash

jupyter notebook

Test data

After running src/train_unet.py, you should have some weights. You can run the follwing step to see your results.

python src/load_weights_and_test.py -w <PATH OF TRAINED MODEL>

Data

The data is taken from ICCV09 [1] which contains 715 images sourced from LabelMe, MSRC, PASCAL VOC, and Geometric Context. Labels were obtained using Amazon's Mechanical Turk (AMT). Data can be downloaded from here

The labels are:

horizons.txt : image dimensions and location of horizon
labels/*.regions.txt : integer matrix indicating each pixel's semantic class (sky, tree, road, grass, water, building, mountain, or foreground object). A negative number indicates unknown.
labels/*.surfaces.txt : integer matrix indicating each pixel's geometric class (sky, horizontal, or vertical).
labels/*.layers.txt : integer matrix indicating distinct image regions.

What is UNet?

UNet[2] is Convolutional Neural Network (CNN) which builds on the well known FCN[3]. Compared to the traditional sliding window approach, which won the Electron Microscopy Segmentation challenge at ISBI 2012, the FCN pose a more elegant architecture which reduces redundant overlapping patches its predecessor had.

The Unet consists of 23 convolutional layers with one contraction and one, more or less symmetric, expansion path. A concatenation of high resolution features from the contracting path to the unsampled features from the expanding path allows for localization.

The UNet architecture can be seen below

Training

In this repo, we would be using a small version of UNet and train it over 50 epoch. The accuracy_loss_plot is shown below

Accuracy plot

Loss plot

Reference

[1] Decomposing a Scene into Geometric and Semantically Consistent Regions, Stephen Gould, Richard Fulton, Daphne Koller, ICCV 2009

[2] U-Net: Convolutional Networks for Biomedical Image Segmentation. Olaf Ronneberger, Philipp Fischer, Thomas Brox

[3] Fully Convolutional Networks for Semantic Segmentation. Jonathan Long, Evan Shelhamer, Trevor Darrell. UN Berkeley

piperod / UNet