This package is a Reinforcement Learning package written in LUA for Torch. It main features are (for now):
- Different environments are provided, from classical RL environments, ATARI games, to special ones like the
multiclass classification environment
that casts a classification learning problem to a RL problem.- Classic reward-based policies: Policy gradient, recurrent policy gradient, approximated q-learning with experience replay (also known as deep Q learning)
- Imitation-based policies: Stochastic gradient-based imitation policy
- Predictive policies: policies which goal is to predict an output and not to maximize a reward
- The different policies can be easily used with
openAI Gym
directly in python by using thelutorpy
package
- 19th of October 2016: OpenAI works again....
- 19th of August 2016: Major update !
- Now, environments have been splitted in three components:
World
,Sensor
andFeedback
. It allows one to easily specify different sensors, and different problems on the same world. It also greatly increase the readability of the platform - Tutorials have been updated
- WARNING: the client/server classes also have to be updated
- Now, environments have been splitted in three components:
Lua:
- Torch7
- nn, dpnn
- logroll, json, alewrap, sys, paths, tds
luarocks install nn
luarocks install dpnn
luarocks install logroll
luarocks install json
luarocks install sys
luarocks install paths
luarocks install tds
git clone https://github.com/deepmind/xitari.git && cd xitari && luarocks make && cd .. && rm -rf xitari
git clone https://github.com/deepmind/alewrap.git && cd alewrap && luarocks make && cd .. && rm -rf alewrap
For using openAI Gym:
- openai gym
- lutorpy
cd torch && luarocks make
- Install lutorpy and OpenAI Gym
- lauch the python script (example.py)
The package if composed of these different elements:
- Core: the core classes
- Sensors: the different generic sensors (that are not specific to a particular world)
- Policies: different (learning) policies
- Environments: different environments
- Classic Control Tasks: Classic control tasks
- Atari: Atari environments
- Classic Machine Learning: We also provide some environments that correspond to classical machine learning problems seen as RL environments
- Tools: different tools
(To update) The interface with the open AI Gym package is explained Here
The tutorials are avalaible here: Tutorials
- When installing Lutorpy, Luajit is not being detected.
Check that pkg-config can find luajit. The following should return at least one result:
pkg-config --list-all | grep luajit
If there are no results, then your .pc
file for luajit is probably not in the right place. Try something like the following:
ln -s /path/to/torch/exe/luajit-rocks/luajit-2.0/etc/luajit.pc /usr/local/lib/pkgconfig/luajit.pc
- Exception related to no display, such as
pyglet.canvas.xlib.NoSuchDisplayException: Cannot connect to "None"
.
OpenAI Gym needs some sort of display to record results. On ubuntu, you may try to install xvfb/asciinema. Then try running example.py like so:
xvfb-run -s "-screen 0 1400x900x24" python example.py
Author: Ludovic DENOYER -- The code is provided as if, some bugs may exist.....