TF-Agents: A library for Reinforcement Learning in TensorFlow

NOTE: Current TF-Agents pre-release is under active development and interfaces may change at any time. Feel free to provide feedback and comments.

The documentation, examples and tutorials will grow over the next few weeks.

Agents
Tutorials
Examples
Installation
Contributing
Principles
Citation
Disclaimer

Agents

In TF-Agents, the core elements of RL algorithms are implemented as Agents. An agent encompasses two main responsibilities: defining a Policy to interact with the Environment, and how to learn/train that Policy from collected experience.

Currently the following algorithms are available under TF-Agents:

DQN: Human level control through deep reinforcement learning Mnih et al., 2015 https://deepmind.com/research/dqn/
DDQN: Deep Reinforcement Learning with Double Q-learning Hasselt et al., 2015 https://arxiv.org/abs/1509.06461
DDPG: Continuous control with deep reinforcement learning Lilicrap et al. https://arxiv.org/abs/1509.02971
TD3: Addressing Function Approximation Error in Actor-Critic Methods Fujimoto et al. https://arxiv.org/abs/1802.09477.
REINFORCE: Simple Statistical Gradient-Following Algorithms for Connectionist Reinforcement Learning Williams http://www-anw.cs.umass.edu/~barto/courses/cs687/williams92simple.pdf
PPO: Proximal Policy Optimization Algorithms Schulman et al. http://arxiv.org/abs/1707.06347

Tutorials

See tf_agents/colabs/ for tutorials on the major components provided.

Examples

End-to-end examples training agents can be found under each agent directory. e.g.:

DQN: tf_agents/agents/dqn/examples/train_eval_gym.py

Installation

Stable Builds

To install the latest version, run the following:

# Installing with the `--upgrade` flag ensures you'll get the latest version.
pip install --user --upgrade tf-agents  # depends on TensorFlow

TF-Agents depends on a recent stable release of TensorFlow (pip package tensorflow).

Note: Since TensorFlow is not included as a dependency of the TF-Agents package (in setup.py), you must explicitly install the TensorFlow package (tensorflow or tensorflow-gpu). This allows us to maintain one package instead of separate packages for CPU and GPU-enabled TensorFlow.

To force a Python 3-specific install, replace pip with pip3 in the above commands. For additional installation help, guidance installing prerequisites, and (optionally) setting up virtual environments, see the TensorFlow installation guide.

Nightly Builds

There are also nightly builds of TF-Agents under the pip package tf-agents-nightly, which requires you install on one of tf-nightly and tf-nightly-gpu. Nightly builds include newer features, but may be less stable than the versioned releases.

To install the nightly build version, run the following:

# Installing with the `--upgrade` flag ensures you'll get the latest version.
pip install --user --upgrade tf-agents-nightly  # depends on TensorFlow

Contributing

We're eager to collaborate with you! See CONTRIBUTING.md for a guide on how to contribute. This project adheres to TensorFlow's code of conduct. By participating, you are expected to uphold this code.

Principles

This project adheres to Google's AI principles. By participating, using or contributing to this project you are expected to adhere to these principles.

Citation

If you use this code please cite it as:

@misc{TFAgents,
  title = {{TF-Agents}: A library for Reinforcement Learning in TensorFlow},
  author = "{Sergio Guadarrama, Anoop Korattikara, Oscar Ramirez,
    Pablo Castro, Ethan Holly, Sam Fishman, Ke Wang, Katya Gonina,
    Chris Harris, Vincent Vanhoucke, Eugene Brevdo}",
  howpublished = {\url{https://github.com/tensorflow/agents}},
  url = "https://github.com/tensorflow/agents",
  year = 2018,
  note = "[Online; accessed 30-November-2018]"
}

Disclaimer

This is not an official Google product.

GitHub30 / agents