Recording video on a server

[justheuristic]

Greetings!

I am trying to run this on a server (accessed via ssh)

save_path = '/tmp/whatever'

subm_env = gym.make(GAME_TITLE)

subm_env.monitor.start(save_path,force=True)

observation = subm_env.reset()

action = 0

observation, reward, done, info = subm_env.step(action)

And it returns this error

[2016-04-30 22:53:30,482] Making new env: SpaceInvaders-v0
[2016-04-30 22:53:30,512] Creating monitor directory /tmp/whatever
[2016-04-30 22:53:30,527] Starting new video recorder writing to /tmp/whatever/openaigym.video.13.18369.video000000.mp4

---------------------------------------------------------------------------
IOError                                   Traceback (most recent call last)
<ipython-input-42-16c8f8e17e6b> in <module>()
     10 action = 0
     11 
---> 12 observation, reward, done, info = subm_env.step(action)

/home/jheuristic/yozhik/gym/gym/core.pyc in step(self, action)
     80         self.monitor._before_step(action)
     81         observation, reward, done, info = self._step(action)
---> 82         done = self.monitor._after_step(observation, reward, done, info)
     83         return observation, reward, done, info
     84 

/home/jheuristic/yozhik/gym/gym/monitoring/monitor.pyc in _after_step(self, observation, reward, done, info)
    209         self.stats_recorder.after_step(observation, reward, done, info)
    210         # Record video
--> 211         self.video_recorder.capture_frame()
    212 
    213         return done

/home/jheuristic/yozhik/gym/gym/monitoring/video_recorder.pyc in capture_frame(self)
    109                 self._encode_ansi_frame(frame)
    110             else:
--> 111                 self._encode_image_frame(frame)
    112 
    113     def close(self):

/home/jheuristic/yozhik/gym/gym/monitoring/video_recorder.pyc in _encode_image_frame(self, frame)
    159 
    160         try:
--> 161             self.encoder.capture_frame(frame)
    162         except error.InvalidFrame as e:
    163             logger.warn('Tried to pass invalid video frame, marking as broken: %s', e)

/home/jheuristic/yozhik/gym/gym/monitoring/video_recorder.pyc in capture_frame(self, frame)
    282             raise error.InvalidFrame("Your frame has data type {}, but we require uint8 (i.e. RGB values from 0-255).".format(frame.dtype))
    283 
--> 284         self.proc.stdin.write(frame.tobytes())
    285 
    286     def close(self):

IOError: [Errno 32] Broken pipe

Is there any way to record video in my setup?

p.s. the environment is Atari Space Invaders and the code is here

p.c. on a previous recently closed issue - any comments on how to make tutorial code more readable? (since that was the purpose of the issue in the first place)

[Greg Brockman]

I'm slightly surprised that there isn't a nicer error message for you, but does using the xvfb command recommended here help: https://gist.github.com/joschu/e42a050b1eb5cfbb1fdc667c3450467a?

I'll re-open the other issue so we can chat there about readability. Sorry, think we missed that the first time around!

[Trevor Blackwell]

Most likely ffmpeg is failing. Can you try this:

dd if=/dev/zero bs=750000 count=50 | ffmpeg -nostats -loglevel error -y -r 60 -f rawvideo -s:v 500x500 -pix_fmt 'rgb24' -i /dev/stdin -vcodec libx264 -pix_fmt yuv420p /tmp/foo.mp4

and see if it's able to generate a /tmp/foo.mp4 video file (of all black pixels). If not, the error message should give a clue.

[justheuristic]

Thanks a lot. It said id doesn't know about libx264.

dd if=/dev/zero bs=750000 count=50 | ffmpeg -nostats -loglevel error -y -r 60 -f rawvideo -s:v 500x500 -pix_fmt 'rgb24' -i /dev/stdin -vcodec libx264 -pix_fmt yuv420p /tmp/foo.mp4
Unknown encoder 'libx264'

This did not help (still getting the same error)

sudo apt-get update
sudo apt-get install libav-tools
sudo apt-get install libavcodec-extra-53

However, doing the same thing with avconv worked

dd if=/dev/zero bs=750000 count=50 | avconv -nostats -loglevel error -y -r 60 -f rawvideo -s:v 500x500 -pix_fmt 'rgb24' -i /dev/stdin -vcodec libx264 -pix_fmt yuv420p /tmp/foo.mp4
50+0 records in
50+0 records out
37500000 bytes (38 MB) copied, 0,141512 s, 265 MB/s

Yet the error in the game monitoring stays the same.
It may actually be due to weirdness of that server, so if you have no idea why such behaviour may happen and no one can reproduce it, i believe it's worth simply closing the thing.

[Trevor Blackwell]

The video_recorder logic is

        if distutils.spawn.find_executable('ffmpeg') is not None:
            self.backend = 'ffmpeg'
        elif distutils.spawn.find_executable('avconv') is not None:
            self.backend = 'avconv'

Try deleting the ffmpeg executable so it will use avconv instead. Use:
sudo apt-get remove ffmpeg

I suspect we should prefer avconv over ffmpeg, since it is more predictable.

[justheuristic]

Thank you, that finally worked 👍

[Daniel Seita]

This error also happens on OS X 10.11.6 (El Capitan). In that case, the solution is to reinstall ffmpeg with the -x265 (not -x264...) library:

brew install ffmpeg --with-fdk-aac --with-ffplay --with-freetype --with-libass --with-libquvi --with-libvorbis --with-libvpx --with-opus --with-x265

(From the ffmpeg docs, though I had to get rid of --with-libass to get this to work.)

[Miguel Morales]

In 2019, the solution by @sytelus down 2 comments is what worked for me. He has exact versions which work well! Leaving this comment for context only

For those using conda (and my future self,) I solved a similar issue by:

1. Installing x264:
   conda install x264

2. Upgrading ffmpeg:
   conda install ffmpeg==4.0

[Aviv A. Rosenberg]

In case this helps someone, this issue got resolved for me by specifying ffmpeg>4.1 in my environment.yml file (using conda). It installed 4.1.3. My gym version is 0.12.1.

Before this I had ffmpeg 4.0 which didn't work.

[Shital Shah]

I got this error with Ubuntu 18.04 while using Monitor class in OpenAI Gym. You need correct versions of ffmpeg and x264 encoder. To get them, I used following and it worked:

conda install x264=='1!152.20180717' ffmpeg=4.0.2 -c conda-forge

[hagh]

I get the same error with Ray RLlib when setting the video recording on.
I tried both methods, i.e. conda install x264=='1!152.20180717' ffmpeg=4.0.2 -c conda-forge (and also the newer versions), as well as installing x265 instead of x264.
Both method worked for me in the sense that I don't get the error any more. However, many of the recorded videos are corrupted and cannot be played. I am wondering if people may have experienced the same problem? any suggestion to fix this later issue?

EDIT: Source of my problem should be something else. I tested these in a container and all worked fine.

[Aviv A. Rosenberg]

I encountered this issue again after updating packages.
It works with: ffmpeg=4.2.2, x264=1!157.20191217, gym=0.17.3.
The problem occurred with ffmpeg=4.3 and disappeared when downgrading to 4.2.2.

[Nate Pham]

I got this error with Ubuntu 18.04 while using Monitor class in OpenAI Gym. You need correct versions of ffmpeg and x264 encoder. To get them, I used following and it worked:

conda install x264=='1!152.20180717' ffmpeg=4.0.2 -c conda-forge

This works for gym==0.18.3 python==3.7.10 torch=1.7.1 with Ubuntu 20.04.