ei-grad / track

Autonomous Car Simulation with Reinforcement Learning

This project is an autonomous car simulation built with Python, utilizing reinforcement learning for training an agent to navigate through a predefined track. The simulation integrates several libraries, including Box2D for physics, Pygame for rendering, and TensorForce for implementing reinforcement learning algorithms.

Getting Started

To run this simulation, you will need Python installed on your machine along with the necessary libraries mentioned in the requirements.txt file. This guide assumes you have a basic understanding of Python and virtual environments.

Prerequisites

• Python 3.7
• pip

Installation

1. Clone the repository to your local machine.
2. Create a virtual environment:

   python -m venv venv

3. Activate the virtual environment:
   • On Windows:

     .\venv\Scripts\activate

   • On Unix or MacOS:

     source venv/bin/activate

4. Install the required dependencies:

   pip install -r requirements.txt

Files Overview

• agent.py: The main script where the reinforcement learning agent is set up and trained.
• dqn.json, ppo.json: Configuration files for the DQN and PPO agents, respectively.
• main.py: Contains the core functionality for the simulation, including car dynamics and track rendering.
• requirements.txt: Lists all the necessary Python libraries.
• top_down_car.py: Defines the car model and its behavior.

Running the Simulation

To start the simulation, run the following command:

CONFIG=ppo.json python agent.py

This will initialize the simulation window and start the training process for the autonomous car. You can observe the car's performance as it learns to navigate the track over time.

How It Works

The simulation uses reinforcement learning to train an agent (the car) to complete laps around a track as fast as possible without colliding with the track boundaries. The agent receives observations from the environment, such as its speed, orientation, and distance to track boundaries, and learns to take actions that maximize its cumulative reward over time.

The agent.py script integrates the TensorForce framework to define and train the reinforcement learning agent using configurations specified in dqn.json or ppo.json for DQN or PPO algorithms, respectively.

Customization

You can customize the simulation by modifying the track or adjusting the reinforcement learning parameters:

• Track Customization: Edit the track.svg file to create custom tracks. The simulation will render and adapt to the new track layout.
• RL Parameters: Adjust the settings in dqn.json or ppo.json to experiment with different reinforcement learning algorithms or parameters.

Contributing

Contributions are welcome! Please feel free to submit pull requests or open issues to improve the project.

License

This project is licensed under the MIT License - see the LICENSE file for details.