Reinforcement Learning as a Treasure Hunt 🏝️

Introduction

Ahoy there, adventurer! 🏴‍☠️

In Reinforcement Learning as a Treasure Hunt, you are tasked with collecting as many coins as you can within a limited number of turns. The coins are buried on a desert island, hidden beneath the sand. However, they seem to follow a pattern in how they are buried, related to the many landmarks scattered around the island. Can you figure out the pattern, and use this to your advantage?

This game seeks to teach players about the Exploration/Exploitation Dilemma, a key concept in reinforcement learning. The idea is that an agent should try to explore the unknown environment to learn which actions will give it the highest expected reward (in this case, the number of coins it collects), but it must also take the opportunity to extract this reward; since the agent has a limited number of actions, it must strike a balance between these goals!

The game was originally designed to be played by children aged 5-13, but we've found that it was enjoyed by players of all ages.

The game can also be run as an interactive activity perfect for family-friendly events. We ran the game as an activity at the Oxford Maths Festival 2024 to great success.

Installation

NOTE: You must have MATLAB installed on your machine to run this game.

To install the game, simply clone this repo into a directory accessible by MATLAB.

Note that this game uses the Papyrus font, but if you do not have this font installed on your machine, it will use the default MATLAB font instead.

Starting a Game

In the MATLAB Editor, make sure you are in this repo's directory on your machine, open the file Game.m and press the green 'Run' button to start. This should load up the game window.

You can make the game appear larger on the screen by maximizing the game window.

How to Play

To play the game, simply click on the tiles in the grid. Once the player has run out of turns, close the game window to exit the game. Currently there is no 'restart' button, and you must run the game again to play another round.

Tips for Getting a Good Score

The coins spawn within 3x3 boxes centred on a certain type of landmark on the map (selected at random each game). To get a good score, the player should figure out which type of landmark marks the treasure. Note that you can select tiles with landmarks in them, and the correct landmark always gives a coin when selected.

The coins also spawn in the same pattern within the 3x3 boxes (again, randomly generated each game); if the player figures out how the treasure is buried around one landmark, they also know how it's buried around other landmarks of the same type.

Note that, even when knowing exactly how the game works and playing optimally, it is very unlikely that you will get the maximum possible score!

Changing Game Parameters

The top of the file Game.m contains some parameters that can be tuned to make the game easier or harder, such as increasing the number of landmarks or decreasing the number of turns.

We found the current setup to work well for an audience of children aged 5-13.

The same file also defines the size and placement of all the graphical elements. If you encounter graphical issues with certain elements, such as the row and column labels for the grid (as I did when running the game on a different laptop), we suggest changing their size and location until you're happy with the result.

Running as an Interactive Activity at an Event

We recommend using a projector and screen to present the game. this helps the activity stand out from a distance and means that participants don't need to crowd around a small screen.

The activity runs more smoothly if the participant calls out their chosen tile (using the row and column labels) for a dedicated game operator to select, rather than giving the participant direct control. This is especially the case for younger participants.

The game benefits from having two game operators at a time; one to operate the game from a laptop while the other interacts with the participant (perhaps giving them hints if they are stuck).

We also suggest that you reward participants with a prize after playing; we offered chocolate gold coins as well as plastic coins and gems. It looks appealing if you pile them up in a 'treasure hoard' on a table near the screen.

If you want to go the extra mile, we suggest wearing pirate hats and playing some piratical background music!

Pre/Post-Game Script

We provide an example script, for introducing the game to new players as well as discussing what they've learned after playing. The idea is that we want to introduce the rules of the game in an engaging and approachable way, as well as giving some hints to point them in the right direction. After the player has completed the game, we take the opportunity to draw a relation to reinforcement learning (again, in a way that is approachable to children).

Credits

Lead Game Designers: Jacques Cloete, Harry Mead

Software Developers: Jacques Cloete, Harry Mead

Maintainer: Jacques Cloete

Documentation Author: Jacques Cloete

Pre/Post-Game Script Writer: Darius Muglich

We thank Luisa Kurth, Shozen Dan, Paula Cordero Encinar, Marcel Hedman and Rafael Brutti for their suggestions that helped to shape the design of the game.

Attributions for Graphics

The island background graphic was designed by Jacques Cloete, with the use of DeepAI for a first draft.

The shadow and grid graphics were made by Jacques Cloete.

All other graphics were designed by FreePik:

JacquesCloete / RL_Treasure_Hunt