TheFebrin / Evolutionary-cars

Cars learn how to drive using evolutionary algorithms and neural networks.

Evolutionary-cars

Cars learn how to drive using evolutionary algorithms and neural networks.

Table of contents

• General info
• Neural network
• Objective function
• Genetic algorithm
• Screenshots
• Technologies
• Libraries
• Results
• Status

General info

Cars:

1. Each car has k sensors.
2. Car's position is calculated using its current velocity and angle.
3. Each sensor detects the distance from the car to the race track barrier.
4. Those measurements are inputs of the neural network that predicts the action.
5. The neural network learns using an evolutionary algorithm.
6. Car's velocity and angle are changed according to the network's output.

Tracks:

• Track 1 - easy
• Track 2 - hard
• Track 3 - very hard - waiting for introduction

Miscellaneous:

1. Program saves the best cars it has seen.
2. Program can save all sensors measurements during the laps.

Neural network

I used a simple network with 2 hidden layers.

• Input Layer: (k sensors measurements, actual speed, actual angle)

• Output Layer: (angle change, acceleration change)

Objective function

As it's quite hard to define an objective function I created checkpoints.

• When a car reaches a checkpoint it earns points.
• It is punished for the distance to the next checkpoint.

Genetic algorithm

1. Parent Selection

   From the current population, we choose parents such that the probability of choosing a parent p is equal to its fitness value f(x), where:

   

   F(x) is an objective function value.

2. Crossovers

   Parents chosen in the previous step pair up to produce offspring.

   • Simple crossover

     Each pair produces two children. We draw n such pairs, where n is a population size
     and do as follows:

     1. Iterate through genotype.
     2. With 50% probability, child_1 will get a gene from parent_1, then child_2 gets a gene from parent_2.
     3. Otherwise child_1 will get a gene from parent_2 and child_2 from parent_1.

   • Neural networks crossover

     Each pair produces one child. We draw n such pairs, where n is a population size.

     We have two parent neural networks N1 and N2. The goal is to produce an offspring network N3. We do as follows:

     1. For each car we record the sensors measurements and car's decision at every frame.
     2. From each parent we draw k samples of those measurements and corresponding predicts.
     3. We feed 2k samples to the child network N3.
     4. We hope that N3 learned something and drives not worst than it's parents.

3. Mutations

   • Simple mutation

     Add a Gaussian Noise to the network.

   • ES mutation

     Use ES(λ + μ) algorithm for mutation. This is quite powerful algorithm, which gives us a possibility to run it alone without a crossover.

Screenshots

• Sensor

  

• Track 1

  

• Track2

  

• Track 3

  

Technologies

• Python - version 3.7.3

Libraries

• numpy
• pygame
• Image
• pytorch
• json

Results

• Cars have no problems with learning how to deal with the first track.
• For now they cannot finish 2nd track, because they speed up to much near the end.
• Neural network crossover works quite well, but it is slow.
• Simple and fast ES is the best way to go here.

Status

Project is: in_the_middle

Credits

Created by @TheFebrin
Racing tracks Designed by Freepik