linhlpv / Skycar-System-Integration

Self Driving car Udacity system integration projects. Team Skycar. 5 engineers from Europe.

Team Skynet - self-driving car system integration

Mission: Control CARLAs (self driving car) throttle, steering and brakes to successfully navigate map waypoints and to detect traffic lights, stopping if the light is red.

Before running our code on CARLA we developed it to work in a simulator. The simulator works in a very similar way to CARLA as all ROS nodes and topics are then same. So basically if our system works in the simulator, we expect that it should also work on CARLA. This is with the exception of obstacles (e.g. traffic light) detection, as different classification model is used because of pixel inputs. To start using this repo please review first this section.

Running Skycar

When the simulator, ROS and catkin environment are up and running, you should be able to run this project.

1. Navigate to repository

2. Navigate to ros directory: cd ros

3. Run

   • Simulator: ./run_skycar_styx
   • CARLA (site): ./run_skycar_site

Self Driving car basic system architecture. Principle behind systems architecture.

CARLA

CARLAs ROS Nodes:

1. Perception

   • Traffic light detection - We use a deep neural net to detect if the upcoming traffic light is red or not. We trained the classifier once with images from the simulator and once with real images from the ROS bag. A detailed description of the architecture and training parameters can be found in the respective notebooks here and here.

2. Planning

   • Waypoint Updater - sets target velocity for each waypoint based on upcoming traffic lights and obstacles. More detailed documentation can be found here.

3. Control subsystems

   • DBW (Drive By Wire) - takes target trajectory information as input and sends control commands to navigate the vehicle. More detailed documentation can be found here.

The Team

Image	Role	Name	Location	LinkedIn	email
	Lead	Nauris Dorbe	Latvia, Riga	Nauris	[e-mail]
		Neil Maude	UK, Leeds	Neil	[e-mail]
		Jonas Biehler	Germany, Munich	Jonas	[e-mail]
		Martin Liesenberg	Germany, Berlin	-	[e-mail]
		Alexander Noll	Austria, Vienna	-	[e-mail]

Nauris Dorbe: Machine Learning Expert at SqualioCC, Researcher assistant at EDI and Doctor student at University of Latvia. Interested in Machine Learning and Self Driving cars. Github. LinkedIn. Twitter

Neil Maude: ([e-mail]): General Manager at Arena Group EDM. MA (Math/Computing), University of Oxford. MBA, Warwick Business School. Interested in AI, Machine Learning, Data Visualisation and Self Driving Cars/Autonomous Vehicles. Github. LinkedIn. Twitter.

Martin Liesenberg: Software Engineer at BCG Digital Ventures. MsC in Computer Science from FU Berlin. Interested in and working with machine learning, stream processing and autonomous vehicles. Github

Alexander Noll: Data Scientist at PartnerRe. PhD in Mathematics from University of Vienna, MSc in Engineering Physics from TU Graz. Interested in machine learning, statistics, artificial intelligence and self-driving cars. Github

Jonas Biehler: Research Engineer at AdCo Engineering. Phd in Computational Mechanics. Interested in Simulation, AI, Machine Learning, HPC, and self-drinving cars. Github, LinkedIn

Project details

This is the project repo for the final project of the Udacity Self-Driving Car Nanodegree: Programming a Real Self-Driving Car. For more information about the project, see the project introduction here.

Installation

• Be sure that your workstation is running Ubuntu 16.04 Xenial Xerus or Ubuntu 14.04 Trusty Tahir. Ubuntu downloads can be found here.

• If using a Virtual Machine to install Ubuntu, use the following configuration as minimum:

  • 2 CPU
  • 2 GB system memory
  • 25 GB of free hard drive space

  The Udacity provided virtual machine has ROS and Dataspeed DBW already installed, so you can skip the next two steps if you are using this.

• Follow these instructions to install ROS

  • ROS Kinetic if you have Ubuntu 16.04.
  • ROS Indigo if you have Ubuntu 14.04.

• Dataspeed DBW

  • Use this option to install the SDK on a workstation that already has ROS installed: One Line SDK Install (binary)

• Download the Udacity Simulator.

Docker Installation

Install Docker

Build the docker container

docker build . -t capstone

Run the docker file

docker run -p 127.0.0.1:4567:4567 -v $PWD:/capstone -v /tmp/log:/root/.ros/ --rm -it capstone

Usage

1. Clone the project repository

git clone https://github.com/udacity/CarND-Capstone.git

2. Install python dependencies

cd CarND-Capstone
pip install -r requirements.txt

3. Make and run styx

cd ros
catkin_make
source devel/setup.sh
roslaunch launch/styx.launch

4. Run the simulator

Real world testing

1. Download training bag that was recorded on the Udacity self-driving car (a bag demonstraing the correct predictions in autonomous mode can be found here)

2. Unzip the file

unzip traffic_light_bag_files.zip

3. Play the bag file

rosbag play -l traffic_light_bag_files/loop_with_traffic_light.bag

4. Launch your project in site mode

cd CarND-Capstone/ros
roslaunch launch/site.launch

5. Confirm that traffic light detection works on real life images