C++ code implementing a Markov Decision Process. ATTENTION: This is not the final version, it will be subject to changes and adjustments in the code and eventually organization of the classes.
For this code I created three classes:
-
Action: It represents an Action that an agent can execute. This class may be inherited in order to create more specific actions ( (for example, we can create Up, Down, Left, Right classes for a robot agent acting in a certain environment.)
-
State: It represents a simple state in the MDP containg a vector of pointer to Actions, a Name and an Id.
-
MDP: This is the main class, containing various data structures representing the states of the MDP, the rewards for every state, the transition function, the optimal policy. The main methods for this class are 'computePolicy()', that is needed in order to compute the optimal policy for the MDP using a Dynamic Programming algorithm, and 'step()', that executes a probabilistical step for the MDP.
-
MDPFactory: Facility singleton class returning instances of MDPs as shared pointers.
(In this example rewards are chosen in this way: S0->5, S1->10, S2->6))
Just:
make
The executable file will be stored in the make directory.
- Try to make a visual representation of the MDP for every step taken.
- Use factory patterns in order to centralize the creation of the objects. ☑