This project is a part of:
The goals of this project are the following:
- Compute the camera calibration matrix and distortion coefficients given a set of chessboard images.
- Apply a distortion correction to raw images.
- Use color transforms, gradients, etc., to create a thresholded binary image.
- Apply a perspective transform to rectify binary image ("birds-eye view").
- Detect lane pixels and fit to find the lane boundary.
- Determine the curvature of the lane and vehicle position with respect to center.
- Warp the detected lane boundaries back onto the original image.
- Output visual display of the lane boundaries and numerical estimation of lane curvature and vehicle position.
This is description of how the processing is made with sample results on two sample input frames.
-
Take RAW image
-
Undistort image (after processing sample chessboard images on given camera)
-
Binarize image using color transforms & magnitude of sobels (image was grayscaled to sum of half RED channel of RGB color space and half SATURIATION channel of HLS color space. Then gradient magniture of x&y solbels was taken and finally binary added to RED & SATURATION channels multiplication)
-
Apply trapezoidal Region of Interest (ROI) shape of expected straigt lane
-
Perspective transform of ROI to bird-eye view.
-
Use binary image without ROI highlight for further processing
-
Starting from bottom search for centers of mass inside rectangular areas (while shifting window to new center)
Bottom start place is calculated basing on historam of bottom part of binary lanes image in axis of x like so:
- Basing on centers of lanes inside areas find quadratic function that matches the points and plot the lanes
For video processing: The quadratic function parameters are first averaged on 3 recent frames to get more stable result and overcome minor artefacts on binary image that may appear
-
Perform lane drawing on bird-eye view of the original image
-
Invert perspective transform and merge result with undistorted part of images that were out-of bounds of bird-eye view
-
The radius of lanes has been calculated by applying radius of curvature method. Since the video is recorded in known location and it is a turn of radius approx 1km a linear correlation between pixel radius and meters were calculated using average of values when the car is turning.
-
Basing on left and right lane curves the pixel range of lane is calculated. Knowing the width of lane in real world pixel-to-meters ratio is calculated. Then after calculating the center between lanes off-center vlue is calculated. All calculated values are shown on last frame.
Download Video File 1: output frame with all subsequent stages of processing (as shown above)
Download video file 2: output frame only
- binary mask is vurnerable to some of irregularities on the road
- used perspective wrap may be insufficient for very hard turning
- current algorithm works only well on flas roads - it might be having troubled with hills and slopes
- this algorithm may still have some problems when changing lanes (especially when being between lanes)
- further improvement may be limiting the area of lane search to some margin of what was found on last frame (look ahead filters)
- when calculating centers of windows (when sarching form curves) there might be applied some bounds - like lower and upper limit when the window should be skipped in curve calculation. This is to addres "minor false positive detections" and "too much noise" cases.
- tweaking the arguments (binary mask, search windows parameters) is very challanging task and addressing it with deep learning model with big and diverse dataset will work much better