Pass args and kwargs to environment upon registration

[ogirdorodrigo]

I currently have a game to which I need to pass positional arguments (keyword arguments would also work) when its init function is called. I believe that to do this cleanly, I would have to do it through the "register_with_gym" function. However, such functionality doesn't seem to be implemented at the level of easyagents. Broadly said (in code), I would like to register an environment that looks like the one below:

class MyEnv(gym.Env):
    def __init__(self, *args):
        self.some_attribute = some_function(*args)
        ...

The goal is to try to simultaneously test many environments with different sets of hyper-parameters.

[Christian Hidber]

Hi ogirdorodrigo

excellent idea. I gave it a try in the 'register_kwargs' branch. With that you should now be able to register your env with kwargs like this:
register_with_gym("myenv-v0", MyEnv, a1=1, a2=2)
You may also reregister the env with a new set of args
register_with_gym("myenv-v0", MyEnv, a1=3, a2=4)
Let me know, if that works for you.

All the best, Christian

[ogirdorodrigo]

Thanks! It is working perfectly :)

[ogirdorodrigo]

Hi there,
I just bumped into an error trying to load a trained agent. It is easily solved passing the original set of parameters during registration, before using the load function. I wonder if it would be posible to read them from the easyagent.json??

[Christian Hidber]

Hi ogirdorodrigo
I guess you are thinking about something like this:

register_with_gym("myenv-v0", MyEnv, a1=3, a2=4)
agent = PpoAgent('myenv-v01)
my_dir = agent.save()
------ in a new python instance ----
agent2 = load(my_dir)
agent2.play(....)

expecting, that agent2 runs against myenv-v0 with the same args as during training. This would imply that load implicitely performs a

register_with_gym("myenv-v0", MyEnv, a1=3, a2=4)

I'm somewhat reluctant, since

1. agents so far operate on the registered gym environment, but do not manipulate it. In particular an implicit call to register with new args changes the behaviour of the env for this and for all other current agent instances.
2. To perform the call to register during load we also need access to the current MyEnv type. Passing it along as a load argument seems to me a bit "unexpected"

On the other hand, it might be attractive to store some additional data (some "context") along with the agent. As a simple" solution you might save the kwargs of the register_with_gym call on your own in the my_dir directory (after your call to agent.save). Before your call to load, load your kwargs and register them again alog with the MyEnv type.

[ogirdorodrigo]

Hi Christian,
Indeed I was thinking about a behaviour as you outline in your example. I agree, that it seems odd to include a call to register_with_gym within load; I didn't take this into account. I thought it would have been a good idea to store the environment's kwargs directly with the save method of the agent. As a documentation (context) of the environment on which the agent was trained. However, if there are any intermediate re-register calls that change the settings, perhaps one would have to consider which one to save, or how to save them all (?!).
The alternative solution you propose is simple enough, so maybe for the sake of transparency it is better to keep the agents split from the environments in this sense. Thanks for your answer though!