Prototype of a game where a reinforcement learning agent is trained through natural language instructions.
Requirements:
- Python 3
- OpenAI gym
- numpy
- PyQT5
- PyTorch
- matplotlib
- nltk
Start by manually installing PyTorch.
Then, clone the repository and install the other dependencies with pip3:
git clone https://github.com/maximecb/baby-ai-game.git
cd baby-ai-game
pip3 install -e .
To run the interactive UI application:
./main.py
Offline training code implementing PPO can be run with:
python3 rl/main.py --env-name AI-Game-v0 --no-vis --num-processes 32 --algo ppo
You can find here a presentation of the project : Baby AI Summary
The Baby AI Game is a game in which an agent existing in a simulated world will be trained to complete task through reinforcement learning as well as interactions from one or more human teachers. These interactions will take the form of natural language, and possibly other feedback, such as human teachers manually giving rewards to the agent, or pointing towards specific objects in the game using the mouse.
The goal of the project is to explore ways in which deep learning can take inspiration from nature (ie: how human babies learn), and to make contributions to the field of reinforcement learning. In particular, language learning, as well as teaching agents to complete actions spanning many (eg: hundreds) of time steps, or macro-actions composed of multiple micro-actions, are still open research problems.
Some possible approaches to be explored in this project include meta-Learning and curriculum learning, the use of intrinsic motivation (curiosity), and the use of pretraining to give agents a small core of built-in knowledge to allow them to learn from human agents. With respect to build-in knowledge, Yoshua Bengio believes that the ability for agents to understand pointing gestures in combination with language may be key.
TODO: find child development articles about pointing and naming if possible. If anyone can find this, please submit a PR.
Zero-Shot Task Generalization with Multi-Task Deep Reinforcement Learning
Beating Atari with Natural Language Guided Reinforcement Learning
Agent-Agnostic Human-in-the-Loop Reinforcement Learning
Trial without Error: Towards Safe Reinforcement Learning via Human Intervention
Grounded Language Learning in a Simulated 3D World
Mastering the Dungeon: Grounded Language Learning by Mechanical Turker Descent
Programmable Agents and associated RLSS 2017 talk by Nando de Freitas
FiLM: Visual Reasoning with a General Conditioning Layer
Surprise-Based Intrinsic Motivation for Deep Reinforcement Learning
Count-Based Exploration with Neural Density Models
Deep RL Bootcamp lecture on Policy Gradients
Proximal Policy Optimization (PPO) Algorithms and blog post by OpenAI
Asynchronous Methods for Deep Reinforcement Learning (A3C)
Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networks
Learning to learn by gradient descent by gradient descent
Learning Language Games through Interaction
Nintendogs (Nintendo DS game)
A Roadmap for Cognitive Development in Humanoid Robots