Atari training and LBFGS gpu memory overhead

[Viktor Makoviychuk]

Hi John,

I'm trying to apply TRPO to the robotics control task, using vision. But constantly hit a GPU memory overhead in class NnRegression in fit during baseline calculation. On the input there were one 128x128 greyscale image and 14 joints observations. Overhead can be seen even when I tried smaller number of iterations and switched to the 96x96 image size. Replacing LBFGS optimizer helped to some extent - there were no crashes but convergence and calculation time became worse.

Did you meet similar memory overhead issues during Atari training and if yes how did you solve them? Input in Atari games is at least 4 times larger then in my cases. So stored volume of the observations data in paths should be even larger or at least compared to my case.