PLARD

Progressive LiDAR Adaptation for Road Detection Implemented in PyTorch

This repository reproduces the results of PLARD PDF in PyTorch. The code is heavily based on pytorch-semseg.

Abstract

Despite rapid developments in visual image-based road detection, robustly identifying road areas in visual images remains challenging due to issues like illumination changes and blurry images. To this end, LiDAR sensor data can be incorporated to improve the visual image-based road detection, because LiDAR data is less susceptible to visual noises. However, the main difficulty in introducing LiDAR information into visual image-based road detection is that LiDAR data and its extracted features do not share the same space with the visual data and visual features. Such gaps in spaces may limit the benefits of LiDAR information for road detection. To overcome this issue, we introduce a novel Progressive LiDAR Adaptation-aided Road Detection (PLARD) approach to adapt LiDAR information into visual image-based road detection and improve detection performance. In PLARD, progressive LiDAR adaptation consists of two subsequent modules: 1) data space adaptation, which transforms the LiDAR data to the visual data space to align with the perspective view by applying altitude difference-based transformation; and 2) feature space adaptation, which adapts LiDAR features to visual features through a cascaded fusion structure. Comprehensive empirical studies on the well-known KITTI road detection benchmark demonstrate that PLARD takes advantage of both the visual and LiDAR information, achieving much more robust road detection even in challenging urban scenes. In particular, PLARD outperforms other state-of-the-art road detection models and is currently top of the publicly accessible benchmark leader-board.

Installation

Please follow instructions on pytorch-semseg.

Setup

Setup Dataset

Please setup dataset according to the following folder structure:

PLARD
 |---- ptsemseg
 |---- imgs
 |---- outputs
 |---- dataset
 |    |---- training
 |    |    |---- image_2
 |    |    |---- ADI
 |    |---- testing
 |    |    |---- image_2
 |    |    |---- ADI

The "image_2" folders contain the visual images which can be downloaded from the KITTI.

The "ADI" folders contain the altitude difference images which can be downloaded here for training set and here for testing set.

Usage

Test

Run the test set on KITTI Road dataset using the following command:

python test.py --model_path /path/to/plard_kitti_road.pth

The results in perspective view will be written under "./outputs/results". Follow the guidelines of KITTI to perform evaluation.

A trained model of PLARD can be downloaded here.

Train

Training script is similar to the pytorch-semseg.

Reference

If you find this code useful in your research, please consider citing:

@article{chen2019progressive,
  title={Progressive LiDAR adaptation for road detection},
  author={Chen, Zhe and Zhang, Jing and Tao, Dacheng},
  journal={IEEE/CAA Journal of Automatica Sinica},
  volume={6},
  number={3},
  pages={693--702},
  year={2019},
  publisher={IEEE}
}

fuhao7i / PLARD