We first develop a learning environment based on PyGames~(\href{http://www.pygame.org/}{www.pygame.org}). It is an open-source python 2D gaming framework and hosts 1000+ games of various kinds, including many Atari forks (Pong, Breakout, etc.). Since we have control over environment, we can turn the knobs to make many games more challenging than they usually are.
Pong
Defender
London Bombing
Video converted by ezgif.com