antonpavlov / adv-lanelines

Advanced Lane Finding - Udacity Self-Driving Car Engineer Nanodegree.

adv-lanelines

Advanced Lane Finding - Udacity Self-Driving Car Engineer Nanodegree.

The goal of this project is to build an advanced software pipeline for an automatic recognition of a road surface markings. A simpler version of this project can be found here: https://github.com/antonpavlov/lanelines. This repository strongly relies and contains references to Udacity's self-driving car nanodegree program.

Contents of the repo

<camera_cal> - A folder with calibration images
<support_files> - A folder with images used in README
<test_images> - A folder with test images; Results of processing will be saved there.
<videos> - A folder with the test video; Result will be saved there.
.gitignore - .gitignore for Python.
LICENSE - The MIT license text.
README.md - this file.
lane-finder.py - The script for lane finding.

Environment set-up

Before run a script from this repo, please setup an environment with all required dependencies: https://github.com/udacity/CarND-Term1-Starter-Kit

Download also an input data from Udacity's project repository: https://github.com/udacity/CarND-Advanced-Lane-Lines

Clone this repository somewhere and copy from there the lane-finder.py script only into the CarND-Advanced-Lane-Lines folder. I suggest to use Anaconda.

Reflection

The following approach was suggested during the course:

1. Compute the camera calibration matrix and distortion coefficients given a set of chessboard images.
2. Apply a distortion correction to raw images.
3. Use color transforms, gradients, etc., to create a thresholded binary image.
4. Apply a perspective transform to rectify binary image ("birds-eye view").
5. Detect lane pixels and fit to find the lane boundary.
6. Determine the curvature of the lane and vehicle position with respect to center.
7. Warp the detected lane boundaries back onto the original image.
8. Output visual display of the lane boundaries and numerical estimation of lane curvature and vehicle position.

Technical restrictions and weaknesses

• Calibration coefficients are related to a specific camera used to record images.
• Script in this repo works only with 1280 X 720 images.
• The proposed pipeline has several weaknesses related to lane marks recognition. Some of them are related to lightning conditions, time of the day, and weather (rain and snow); some of them are related to road surface quality.
• The pipeline is not able to do any inference regarding lane marks and vehicle position when part of the lane ahead is obstructed by another car.
• The pipeline relies on existence of the road marking on that particular road under evaluation.
• Unfortunately, most of coefficients of a transforms and a thresholds are specified on empirical basis considering only images, proposed by Udacity.

Examples of processing

Camera calibration - Original image

Camera calibration - Undistorted image

Let's build the following pipeline:

1. Open an image

2. Apply image correction

3. Application of Sobel operator on undistorted image

4. Filter an image by gradient magnitude in both (x and y) directions

5. Filter an image considering gradient orientation

6. HLS color space threshold

7. All thresholds applied together to undistorted image

8. Perspective transform; warp-in image

9. Find lanes in a binary warped image

10. Make curvature calculations; vehicle position and draw results over an original image

Future work

As a future work, it might be the case for an application of a newly proposed Hinton’s Capsule Networks. These networks may be trained to make an inference over lane existence and its bounds taking into account important spatial hierarchies of lane markings, other vehicles and road equipment.

License

Python script lane-finder.py is distributed under the terms described in the MIT license. Please refer to Udacity regarding all other supporting materials.