Giters

google / dopamine

Dopamine is a research framework for fast prototyping of reinforcement learning algorithms.

Home Page:https://github.com/google/dopamine

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

Getting 7 as action for a game with 3 actions

arjung128 opened this issue · comments

commented

I have been trying to train an online agent on the environment FreewayNoFrameskip-v4. Because this gym environment is not deterministic, I seeded the environment. Specifically, in atari_lib.py, I added

  • env.seed(0) after env = gym.make(full_game_name) in create_atari_environment
  • self.environment.seed(0) at the end of the AtariPreprocessing class's __init__ function
  • self.environment.seed(0) at the start of the reset function in the AtariPreprocessing class

No other changes were made. I then used the Batch RL codebase to train an online agent.

In all of training, there was one instance of a 7 stored as the action (specifically the last action at the end of five iterations of training), even though Freeway only has three actions. All other stored actions were {0, 1, 2}. Any ideas what could be the cause of this? Going in and changing this one 7 to the most common action isn't a problem, but if this problem arises repeatedly, and for other games, it could be difficult to deal with.

commented

hi, is this still an issue? are you sure you're reloading the correct checkpoint (for the same game)?
otherwise, it seems like it might make more sense to ask in the batch rl repo?