Keras Deep Learning project to build a self-driving agent that takes camera frames as input and predicts steering, throttle and brake commands, trained using the DeepGTAV self-driving car development environment.
VPilot uses a LRCN (Convolutional Recurrent Neural Network) as self-driving agent, see model.py. It is based on the NVIDIA's autosteering CNN and Long-term Recurrent Convolutional Networks insights. LSTM is used to include also temporal inference in the network, which I believe will help the agent to accurately predict throttle and brake actions also, which contrary to steering angle, are difficult to deduce from single frames.
Training is still in progress but dummy weights can be downloaded here to try drive.py. With train.py onr can easily apply supervised learning with a dataset generated by DeepGTAV to generate its own weights. You just need to change the dataset directories list.
Once proper weights are generated, drive.py can be used to test your agent, allowing DeepGTAV to connect to your model in Reinforcement Learning mode. Once the game is loaded, drive.py will take the current frame from the game, run it through the LRCN and send the output commands back to GTAV and so the vehicle should start moving, hopefully like a real self-driving car :). Don't forget the set the correct arguments when running the script!