This repository can be used to easily train and manage the trainings of imitation learning networks jointly with evaluations on the CARLA simulator. Objectives:
- To enable the user to perform several trainings with a single command.
- To automatically test the trained systems using CARLA.
- Allow the user to monitor several trainings and testings on CARLA with a single glance.
- Allows to perform the testing methodology proposed on the paper "On Offline Evaluation of Vision-based Driving Models"
- Allows to use the models from the paper Exploring the Limitations of Behavior Cloning for Autonomous Driving (paper).
- You can also use a baseline for CARLA Challenge
The COiLTRAiNE framework allows simultaneous training, driving on scenarios in CARLA and prediction of controls on some static dataset. This process can be done on several experiments at the same time.
- Hardware: A computer with a dedicated GPU capable of running Unreal Engine. NVIDIA 1070 or better is recommended.
- OS: Ubuntu also compatible with CARLA (16.04)
To install COiLTRAiNE, we provide a conda environment requirements file. Start by cloning the repository on some folder and then, to install, just run:
conda env create -f requirements.yaml
The first thing you need to do is define the datasets folder. This is the folder that will contain your training and validation datasets
export COIL_DATASET_PATH=<Path to where your dataset folders are>
Download a sample dataset pack, with one training and two validations, by running
python3 tools/get_sample_datasets.py
The datasets; CoILTrain , CoILVal1 and CoILVal2; will be stored at the COIL_DATASET_PATH folder.
To collect other datasets please check the data collector repository. https://github.com/carla-simulator/data-collector
Note: the automatic scenario evaluation only works for CARLA 0.8.x, however you can train and evaluate agents in CARLA 0.9.X.
For doing scenario evaluation in CARLA you must install CARLA 0.8.4 or CARLA 0.8.2 under docker. This tutorial explains how to install CARLA under docker.
Assuming that you have CARLA docker with a docker image name as "carlasim/carla:version" , you can execute the coiltraine system by running:
python3 coiltraine.py --folder sample --gpus 0 -de TestT1_Town01 -vd CoILVal1 --docker carlasim/carla:version
Where the --folder sample is the experiment batch
containing all the experiments that are going to
be trained and validated.
The TestT1 is a driving scenario on Town01, defined as one of the classes on the
drive/suites folder. The validation datasets are passed as parameter with -vd and should be placed
at the COIL_DATASET_PATH folder.
You should see a colored screen on the terminal.
After finishing training and validation, the terminal screen should start driving look like as below.
You will not see any CARLA server screen popping up since CARLA running under docker runs offscreen. Also note that this example trains on sample data and tests on a sample benchmark. Thus, the resulting driving model will be of poor quality. Please, test some of the models from the conditional models zoo to get high performance conditional imitation models.
- Conditional Imitation Learning
- Conditional Imitation Learning CARLA (paper)
- On Offline Evaluation of Vision-based Driving Models (paper)
- Exploring the Limitations of Behavior Cloning for Autonomous Driving (paper)