Road segmentation

Team Member: Jingyuan Ma, Yongqi Wang, Zhi Ye,

This repository contains the tools and models for the the course project of Computational Intelligence Lab (Spring 2019): Road Segmentaion.

Credit:

TorchSeg for the structure of repository

Prerequisites

In our setting, the models are being run inside a deepo Docker container (using the default tag: latest)

A sample workflow using the docker container:

docker pull ufoym/deepo
# change the volume mount before
# mounting existing dir to container is possible with -v /host-dir:/target-dir
docker run -it --name cu100 ufoym/deepo bash
docker ps -a 
# CONTAINER ID        IMAGE               COMMAND             CREATED             STATUS              PORTS               NAMES
# ae2f6abc24d5        ufoym/deepo         "bash"              2 hours ago         Up About an hour    6006/tcp            eager_ganguly
docker ps -a -q
# ae2f6abc24d5
# entering the bash with container ID
docker exec -it cu100 bash
# follow instructions below
git clone https://github.com/wyq977/cil-road-segmentation-2019.git
cd cil-road-segmentation-2019
cd model/cil-resnet50
# train with CUDA device 0
python train.py -d 0
# eval using the default last epoh
python eval.py -d 0 -p ./val
# generate predicted groundtruth
python pred.py -d 0 -p ./pred
# generate submission.csv
python ../../cil-road-segmentation-2019/mask_to_submission.py --name submission -p ./pred/
# submit the submission.csv generated

PyTorch >= 1.0
- pip3 install torch torchvision
Easydict
- pip3 install easydict
tqdm
- pip3 install tqdm
Apex

Detailed Usage

Model dir:

├── config.py
├── dataloader.py
├── eval.py
├── network.py
├── pred.py
└── train.py

Prepare data

Since the training dataset is quite small in size, we have uploaded onto our github repo. In the case of the images are stored in other directory, config.py under model section might be revised accordingly.

With a tab-separated files specifying the path of images and groundtruth, train.txt, val.txt, test.txt.

train.txt or val.txt:

path-of-the-image   path-of-the-groundtruth

Noted that in the test.txt:

path-of-the-image   path-of-the-image

A handy script (writefile.py) using the package glob can be found inside the dataset directory.

Training

To specify which CUDA device used for training, one can parse the index to train.py

A simple use case using the first CUDA device:

python train.py -d 0

Training can be restored from saved checkpoints

python train.py -d 0 -c log/snapshot/epoch-last.pth

Predictive groudtruth labels

Similar to training

python pred.py -d 0 -p ../../cil-road-segmentation-2019/pred/ -e log/snapshot/epoch-last.pth

Evalaute

python pred.py -d 0 -p ../../cil-road-segmentation-2019/val_pred/ -e log/snapshot/epoch-last.pth

Create submission.csv

cd ../../cil-road-segmentation-2019/
python mask_to_submission.py --name submission -p pred/

Distributed Training

All models but u-net support distributed training using torch.distributed.launch

For each run:

export NGPUS=8
python -m torch.distributed.launch --nproc_per_node=$NGPUS train.py

Structure

├── README.md
├── cil-road-segmentation-2019 # datasets and submission script
├── docs
├── utils # helper function and utils for model
├── log
└── model

Under model directory, one can train, predict groundtruth (test images) and evaluate a model, details usage see the usage section above.

Different helpers functions used in constructing models, training, evaluation and IO operations regarding to pytorch could be found under utils folder. Functions or modules adapted from TorchSeg is clearly marked and referenced in the files.

Logistics

Computational resources

Project submission

Submit the final report: https://cmt3.research.microsoft.com/ETHZCIL2019
Signed form here: http://da.inf.ethz.ch/teaching/2019/CIL/material/Declaration-Originality.pdf
Kaggle: https://inclass.kaggle.com/c/cil-road-segmentation-2019

lxxue / cil-road-segmentation-2019