Improving action_space

[Thomas Hirtz]

Some algorithms such as Q-PAMDP needs to know the exact shape of all the action spaces. It would be nice to be able to implement an action space that can give this information. The issue is that in this environment, BREAK has no dimension, it is therefore not possible to copy the technique from https://github.com/cycraig/gym-platform

Need to investigate possible solutions. One may be to give a dummy low and high such as 0 and 0. However, this may break some reinforcement learning algorithms to give a 0 range

import numpy as np
from gym import spaces


ACCELERATE = 0
TURN = 1
BREAK = 2

action_id_to_domain = {
    ACCELERATE: {'low': [0.], 'high': [1.]},
    TURN: {'low': [-1.], 'high': [1.]},
    BREAK: {'low': [], 'high': []},
}

action_space = spaces.Tuple(
    (
        spaces.Discrete(len(action_id_to_domain)),
        spaces.Tuple(
            tuple(
                spaces.Box(low=np.array(d['low']), high=np.array(d['high']), dtype=np.float32)
                for _, d in sorted(action_id_to_domain.items())
            )
        )
    )
)

[Thomas Hirtz]

solved: openai/gym#2692