Useful Commands for Demo
-
Goal Practice with just one player controlled using gamepad and a goal keeper
python3 -m gfootball.play_game --players="gamepad:left_players=1" --level=academy_empty_goal_close
-
Practice against opponents who are lazy and don't move
python3 -m gfootball.play_game --players="gamepad:left_players=1" --level=academy_single_goal_versus_lazy
-
One player on gamepad, one player on keyboard practicing against a lazy team
python3 -m gfootball.play_game --players="gamepad:left_players=1;keyboard:left_players=1" --level=academy_single_goal_versus_lazy
-
Play against an easy agent
python3 -m gfootball.play_game --players "gamepad:left_players=1;ppo2_cnn:right_players=1,policy=gfootball_impala_cnn,checkpoint=/usr/local/google/home/kannappans/Downloads/trained_model_11_vs_11_easy_stochastic"
-
Easy counterattack
python3 -m gfootball.play_game --players="gamepad:left_players=1" --level=academy_counterattack_easy
Google Research Football
This repository contains an RL environment based on open-source game Gameplay Football. It was created by the Google Brain team for research purposes.
This is not an official Google product.
We'd like to thank Bastiaan Konings Schuiling, who authored and open-sourced the original version of this game.
For more information, please look at our paper.
Mailing list: https://groups.google.com/forum/#!forum/google-research-football Comments / suggestions / feature ideas are welcome on this list, and please use it to communicate with us (in addition to GitHub issues).
For non-public matters that you'd like to discuss directly with the Google Research Football team, please use google-research-football@google.com.
Installation
Currently we're supporting only Linux and Python3. You can either install the code from github (newest version) or from pypi (stable version).
-
Install required apt packages with
sudo apt-get install git cmake build-essential libgl1-mesa-dev libsdl2-dev libsdl2-image-dev libsdl2-ttf-dev libsdl2-gfx-dev libboost-all-dev libdirectfb-dev libst-dev mesa-utils xvfb x11vnc libsdl-sge-dev python3-pip
-
Install gfootball python package from pypi:
- Use `pip3 install gfootball`
- This command can run for couple of minutes, as it compiles the C++ environment in the background.
OR install gfootball python package (run the commands from the main project directory):
- `git clone https://github.com/google-research/football.git`
- `cd football`
- `pip3 install .`
- This command can run for a couple of minutes, as it compiles the C++ environment in the background.
Running experiments
Install additional dependencies:
- TensorFlow:
pip3 install "tensorflow<2.0"
orpip3 install "tensorflow-gpu<2.0"
, depending on whether you want CPU or GPU version; - Sonnet:
pip3 install dm-sonnet
; - OpenAI Baselines:
pip3 install git+https://github.com/openai/baselines.git@master
.
Then:
- To run example PPO experiment on
academy_empty_goal
scenario, runpython3 -m gfootball.examples.run_ppo2 --level=academy_empty_goal_close
- To run on
academy_pass_and_shoot_with_keeper
scenario, runpython3 -m gfootball.examples.run_ppo2 --level=academy_pass_and_shoot_with_keeper
In order to train with nice replays being saved, run
python3 -m gfootball.examples.run_ppo2 --dump_full_episodes=True --render=True
Playing the game yourself
Run python3 -m gfootball.play_game
. By default, it starts the
base scenario and the left player is controlled by the keyboard. Different types
of players are supported (gamepad, external bots, agents...). For possible
options run python3 -m gfootball.play_game -helpfull
.
In particular, one can play against agent trained with run_ppo2
script with
the following command:
python3 -m gfootball.play_game --players "keyboard:left_players=1;ppo2_cnn:right_players=1,checkpoint=$YOUR_PATH"
Please note that playing the game is implemented through an environment, so human-controlled players use the same interface as the agents. One important fact is that there is a single action per 100 ms reported to the environment, which might cause a lag effect when playing.
Trained checkpoints
We provide trained PPO checkpoints for the following scenarios:
In order to see the checkpoints playing, run
python3 -m gfootball.play_game --players "ppo2_cnn:left_players=1,policy=gfootball_impala_cnn,checkpoint=$CHECKPOINT" --level=$LEVEL
,
where $CHECKPOINT
is the path to downloaded checkpoint.
Keyboard mapping
The game defines following keyboard mapping (for the keyboard
player type):
ARROW UP
- run to the top.ARROW DOWN
- run to the bottom.ARROW LEFT
- run to the left.ARROW RIGHT
- run to the right.S
- short pass in attack mode, pressure in the defense mode.A
- high pass in attack mode, sliding in the defense mode.D
- shot in the attack mode, team pressure in the defense mode.W
- long pass in the attack mode, goalkeeper pressure in the defense mode.Q
- switch the active player on defense mode.C
- dribble in the attack mode.E
- sprint.
Observations
Environment exposes following raw
observations:
- Ball information:
ball
- [x, y, z] position of the ball.ball_direction
- [x, y, z] ball movement vector.ball_rotation
- [x, y, z] rotation angles in radians.ball_owned_team
- {-1, 0, 1}, -1 = ball not owned, 0 = left team, 1 = right team.ball_owned_player
- {0..N-1} integer denoting index of the player owning the ball.
- Left team:
left_team
- N-elements vector with [x, y] positions of players.left_team_direction
- N-elements vector with [x, y] movement vectors of players.left_team_tired_factor
- N-elements vector of floats in the range {0..1}. 0 means player is not tired at all.left_team_yellow_card
- N-elements vector of integers denoting number of yellow cards a given player has (0 or 1).left_team_active
- N-elements vector of Bools denoting whether a given player is playing the game (False means player got a red card).left_team_roles
- N-elements vector denoting roles of players. The meaning is:0
= e_PlayerRole_GK - goalkeeper,1
= e_PlayerRole_CB - centre back,2
= e_PlayerRole_LB - left back,3
= e_PlayerRole_RB - right back,4
= e_PlayerRole_DM - defence midfield,5
= e_PlayerRole_CM - central midfield,6
= e_PlayerRole_LM - left midfield,7
= e_PlayerRole_RM - right midfield,8
= e_PlayerRole_AM - attack midfield,9
= e_PlayerRole_CF - central front,
- Right team:
right_team
- same as for left team.right_team_direction
- same as for left team.right_team_tired_factor
- same as for left team.right_team_yellow_card
- same as for left team.right_team_active
- same as for left team.right_team_roles
- same as for left team.
- Controlled players information (M is the number of controlled players):
active
- M-elements list of {0..N-1} integers denoting indices of the controlled players. In most common case the list will contain just one element, but in multi-agent setup it will be more.sticky_actions
- M-elements list of 11-elements vectors of 0s or 1s denoting whether corresponding action is active:0
-game_left
1
-game_top_left
2
-game_top
3
-game_top_right
4
-game_right
5
-game_bottom_right
6
-game_bottom
7
-game_bottom_left
8
-game_sprint
9
-game_keeper_rush
10
-game_dribble
- Match state:
score
- pair of integers denoting number of goals for left and right teams, respectively.steps_left
- how many steps are left till the end of the match.game_mode
- current game mode, one of:0
=e_GameMode_Normal
1
=e_GameMode_KickOff
2
=e_GameMode_GoalKick
3
=e_GameMode_FreeKick
4
=e_GameMode_Corner
5
=e_GameMode_ThrowIn
6
=e_GameMode_Penalty
- Screen:
frame
- three 1280x720 vectors of RGB pixels representing rendered screen. It is only exposed when rendering is enabled (render
flag).
Where N
is the number of players on the team.
X coordinates are in the range [-1, 1]
.
Y coordinates are in the range [-0.42, 0.42]
.
Speed vectors represent a change in the position of the object within a single
step.
In addition, environment provides wrappers which convert raw
observations to a different form:
simple115
(aka Simple115StateWrapper) - simplified representation of a game state encoded with 115 floats:- 44 = 11 (players) * 2 (x and y) * 2 (teams) - coordinates of players
- 44 - player directions
- 3 (x, y and z) - ball position
- 3 - ball direction
- 3 - one hot encoding of ball ownership (noone, left, right)
- 11 - one hot encoding of which player is active
- 7 - one hot encoding of
game_mode
extracted
(aka SMMWrapper) - simplified spacial representation of a game state. It consists of several 72 * 96 planes ob bytes, filled in with 0s except for:- 1st plane: 255s represent positions of players on the left team
- 2nd plane: 255s represent positions of players on the right team
- 2nd plane: 255s represent positions of a ball
- 4th plane: 255s represent positions of an active player
- 5th plane (optional, available when
enable_sides_swap
is enabled): either all 0s if playing from the left, or all 255s if playing from the right
pixels
/pixels_gray
(aka PixelsStateWrapper) - pixel representation, downscaled to 72 * 96, and converted to a single grayscale channel forpixels_gray
. In order to use this representation you have to enable rendering in create_environment call (and run on GPU).
For trivial integration with single agent/player learning algorithms we provide a SingleAgentWrapper wrapper, that strips the first dimension of the representation in the case of only one player being controlled.
Scenarios
We provide two sets of scenarios/levels:
-
Football Benchmarks
- 11_vs_11_stochastic - full 90 minutes football game (medium difficulty)
- 11_vs_11_easy_stochastic - full 90 minutes football game (easy difficulty)
- 11_vs_11_hard_stochastic - full 90 minutes football game (hard difficulty)
-
Football Academy - with a total of 11 scenarios
- academy_empty_goal_close - Our player starts inside the box with the ball, and needs to score against an empty goal.
- academy_empty_goal - Our player starts in the middle of the field with the ball, and needs to score against an empty goal.
- academy_run_to_score - Our player starts in the middle of the field with the ball, and needs to score against an empty goal. Five opponent players chase ours from behind.
- academy_run_to_score_with_keeper - Our player starts in the middle of the field with the ball, and needs to score against a keeper. Five opponent players chase ours from behind.
- academy_pass_and_shoot_with_keeper - Two of our players try to score from the edge of the box, one is on the side with the ball, and next to a defender. The other is at the center, unmarked, and facing the opponent keeper.
- academy_run_pass_and_shoot_with_keeper - Two of our players try to score from the edge of the box, one is on the side with the ball, and unmarked. The other is at the center, next to a defender, and facing the opponent keeper.
- academy_3_vs_1_with_keeper - Three of our players try to score from the edge of the box, one on each side, and the other at the center. Initially, the player at the center has the ball and is facing the defender. There is an opponent keeper.
- academy_corner - Standard corner-kick situation, except that the corner taker can run with the ball from the corner.
- academy_counterattack_easy - 4 versus 1 counter-attack with keeper; all the remaining players of both teams run back towards the ball.
- academy_counterattack_hard - 4 versus 2 counter-attack with keeper; all the remaining players of both teams run back towards the ball.
- academy_single_goal_versus_lazy - Full 11 versus 11 games, where the opponents cannot move but they can only intercept the ball if it is close enough to them. Our center back defender has the ball at first.
You can add your own scenarios by adding a new file to the gfootball/scenarios/
directory. Have a look at existing scenarios for example.
Multiagent support
Using play_game script (see 'Playing game yourself' section for details)
it is possible to set up a game played between multiple agents.
players
command line parameter is a comma-separated lists of players for both teams.
For example, to play yourself using a gamepad with two lazy bots on your team
against three bots you can run
python3 -m gfootball.play_game --players=gamepad:left_players=1;lazy:left_players=2;bot:right_players=1;bot:right_players=1;bot:right_players=1
.
Notice the use of left_players=2
for the lazy player, bot player does not support it.
You can implement your own player controlling multiple players by adding its implementation to the env/players directory (no other changes are needed). Have a look at existing players code for an example implementation.
To train a policy controlling multiple players, one has to do the following:
- pass
number_of_players_agent_controls
to the 'create_environment' defining how many players you want to control - instead of calling '.step' function with a single action, call it with an array of actions, one action per player
It is up to the caller to unpack/postprocess it in a desirable way. A simple example of training multi-agent can be found in examples/run_multiagent_rllib.py
Running in Docker
CPU version
- Build with
docker build --build-arg DOCKER_BASE=ubuntu:18.04 --build-arg DEVICE=cpu . -t gfootball
- Enter the image with
docker run -it gfootball bash
GPU version
- Build with
docker build --build-arg DOCKER_BASE=tensorflow/tensorflow:1.12.0-gpu-py3 --build-arg DEVICE=gpu . -t gfootball
- Enter the image with
nvidia-docker run -it gfootball bash
ordocker run --gpus all -it gfootball bash
for docker 19.03 or later.
After entering the image, you can run sample training with python3 -m gfootball.examples.run_ppo2
.
Unfortunately, rendering is not supported inside the docker.
Note that tensorflow/tensorflow:1.12.0-gpu-py3 is based on Ubuntu 16.04 with
python 3.5. If you want python 3.6, you should instead use tensorflow/tensorflow:1.14.0-gpu-py3.
Building a docker image under MacOS
You may need to increase memory for building. Go to the docker menu, then Preferences, then Advanced/Memory and set memory to the 4GB.
Trace dumps
GFootball environment supports recording of scenarios for later watching or
analysis. Each trace dump consists of a pickled episode trace (observations,
reward, additional debug info) and optionally an AVI file with the rendered episode.
Pickled episode trace can be played back later on using replay.py
script.
By default trace dumps are disabled to not occupy disk space. They
are controlled by the following set of flags:
dump_full_episodes
- should trace for each entire episode be recorded.dump_scores
- should sample traces for scores be recorded.tracesdir
- directory in which trace dumps are saved.write_video
- should a video be recorded together with the trace. If rendering is disabled (render
config flag), the video contains a simple episode animation.
There are following scripts provided to operate on trace dumps:
dump_to_txt.py
- converts trace dump to human-readable form.dump_to_video.py
- converts trace dump to a 2D representation video.replay.py
- replays a given trace dump using environment.
Frequent Problems & Solutions
Rendering not working / "OpenGL version not equal to or higher than 3.2"
Solution: set environment variables for MESA driver, like this:
MESA_GL_VERSION_OVERRIDE=3.2 MESA_GLSL_VERSION_OVERRIDE=150 python3 -m gfootball.play_game