SMSnet

Model-evaluator of Publication : "SMSnet: Semantic Motion Segmentation using Deep Convolutional Neural Networks"

Models

The models can be downloaded here: Models This link contains the models trained on City-Kitti-Motion using EFS and without EFS on inf range and 40m range

Test databases

The databases can be downloaded here:

Each compressed archiv includes 8 lmdb databases:

Labels:

IMAGES_LMDB0: Labels with 20m range
IMAGES_LMDB1: Labels with 40m range
IMAGES_LMDB2: Labels with 60m range
LABELS_LMDB: Labels with inf range

Images:

IMAGES_LMDB3: Left image corresponding to label
IMAGES_LMDB4: Previous left image

Flow: Note: The flow is centered at 128 and scaled by 1/6.4 in order to fit into the UC1 LMDB format

IMAGES_LMDB5: Flow with EFS
IMAGES_LMDB6: Flow

Raw Annotations

The full moving car/ static car annotations for the training and validation sets can be downloaded here: Annotations

Prerequisites

Caffe
CUDA
OPENCV
HDF5
PROTOBUF

How to use

Build caffe

This repo comes with a modified version of caffe

Clone repo
Go into extern/modcaffe
Create a build folder "mkdir build && cd build"
Compile: "cmake .." then "make -j8"

SMSnet evaluator

Go to repo and create build dir: "mkdir build && mkdir datasets && mkdir models"
Download KITTI dataset from the provided link and extract it into "datasets"
Download Models and extract them into "models"
Go to build folder: "cd build"
Compile: "cmake .." then "make"
run program : "./caffe_test_ex" . The network and the KITTI database should load. The network predictions and the GT is visualized. To get the next prediction just press Enter. If you want to change the Dataset take a look in the code (main.cpp)

Troubleshooting

If you get : "caffe/proto/caffe.pb.h: No such file or directory" try to go in your caffe dir (extern/modcaffe) and type:

protoc src/caffe/proto/caffe.proto --cpp_out=.
mkdir include/caffe/proto
mv src/caffe/proto/caffe.pb.h include/caffe/proto

Cite

If you use this code or the datasets please make sure that you cite the following papers:

@InProceedings{Vertens17Icra,
author = {Johan Vertens, Abhinav Valada and Wolfram Burgard},
title = {SMSnet: Semantic Motion Segmentation
using Deep Convolutional Neural Networks},
year = 2017,
month = sept,
url = {https://github.com/JohanVer/SMSnet},
address = {Vancouver, Canada}
}

@article{DBLP:journals/corr/CordtsORREBFRS16,
  author    = {Marius Cordts and
               Mohamed Omran and
               Sebastian Ramos and
               Timo Rehfeld and
               Markus Enzweiler and
               Rodrigo Benenson and
               Uwe Franke and
               Stefan Roth and
               Bernt Schiele},
  title     = {The Cityscapes Dataset for Semantic Urban Scene Understanding},
  journal   = {CoRR},
  volume    = {abs/1604.01685},
  year      = {2016},
  url       = {http://arxiv.org/abs/1604.01685},
  timestamp = {Wed, 07 Jun 2017 14:41:02 +0200},
  biburl    = {http://dblp.uni-trier.de/rec/bib/journals/corr/CordtsORREBFRS16},
  bibsource = {dblp computer science bibliography, http://dblp.org}
}

@ARTICLE{Geiger2013IJRR,
  author = {Andreas Geiger and Philip Lenz and Christoph Stiller and Raquel Urtasun},
  title = {Vision meets Robotics: The KITTI Dataset},
  journal = {International Journal of Robotics Research (IJRR)},
  year = {2013}
}

aisrobots / SMSnet