H.264 M2M codec generates garbage for 1920x1200

[Maxim Devaev]

Describe the bug

UPD: It's not a kernel fault, the problem appeared in this firmware commit: raspberrypi/firmware@dc94391

Up to and including kernel 6.1, we on PiKVM had the opportunity to encode a stream of 1920x1200 without any problems. I even sent a patch to support the 5.1 profile for this, it is now accepted here. On kernel 6.6, we started receiving non-periodic garbage in the encoded stream.

Steps to reproduce the behaviour

To reproduce it, you need to use TC358743-based HDMI capture with 4 lanes on CM4. The recommended EDID is here

Compile ustreamer and run it with 1920x1200 HDMI source:

git clone https://github.com/pikvm/ustreamer
cd ustreamer
make -j
./ustreamer --dv-timings --persistent --format=uyvy --h264-sink=foo
<other terminal>
./ustreamer-dump --sink=foo --output - | ffmpeg -use_wallclock_as_timestamps 1 -i pipe: -c:v copy test.mp4

This way you will get test.mp4 file with non-periodic garbage.

Device (s)

Raspberry Pi CM4, Raspberry Pi CM4 Lite

System

• Arch Linux ARM

• Linux pikvm 6.6.21-4-rpi #2 SMP Fri Mar 15 09:58:15 UTC 2024 armv7l GNU/Linux (commit d3d2454)

Copyright (c) 2012 Broadcom
version f4e2138c2adc8f3a92a3a65939e458f11d7298ba (clean) (release) (start)

Logs

No response

Additional context

Here the settings that we used in ustreamer:

        SET_OPTION(V4L2_CID_MPEG_VIDEO_BITRATE,             enc->bitrate);
        SET_OPTION(V4L2_CID_MPEG_VIDEO_H264_I_PERIOD,       enc->gop);
        SET_OPTION(V4L2_CID_MPEG_VIDEO_H264_PROFILE,        V4L2_MPEG_VIDEO_H264_PROFILE_CONSTRAINED_BASELINE);
        if (run->p_width * run->p_height <= 1920 * 1080) { // https://forums.raspberrypi.com/viewtopic.php?t=291447#p1762296
            SET_OPTION(V4L2_CID_MPEG_VIDEO_H264_LEVEL,      V4L2_MPEG_VIDEO_H264_LEVEL_4_0);
        } else {
            SET_OPTION(V4L2_CID_MPEG_VIDEO_H264_LEVEL,      V4L2_MPEG_VIDEO_H264_LEVEL_5_1);
        }
        SET_OPTION(V4L2_CID_MPEG_VIDEO_REPEAT_SEQ_HEADER,   1);
        SET_OPTION(V4L2_CID_MPEG_VIDEO_H264_MIN_QP,         16);
        SET_OPTION(V4L2_CID_MPEG_VIDEO_H264_MAX_QP,         32);

For 1920x1080 we're using 4.0 profile and it's working fine even on 6.6 kernel. The problem only with 1920x1200.

[Maxim Devaev]

It seems this is related with bootloader release. I'll bisect it and find the commit with breaking changes.

[Maxim Devaev]

Okay, I found the commit that brokes everything: raspberrypi/firmware@dc94391

Before this, H.264 1920x1200 was working fine.

[6by9]

Nothing in that commit relates directly to video encode.
The SDRAM change has the potential to affect many things, and there is a slightly suspicious looking block that I will check out.

Your corruption looks to be in the H264 block column format having an incorrect column stride.

Sorry, I'm not going to try setting up your system.
Trying to reproduce through GStreamer by transcoding seems OK

gst-launch-1.0 -evvvv filesrc location=bbb_sunflower_1080p_60fps_normal.mp4 ! qtdemux ! h264parse ! v4l2h264dec ! v4l2convert ! video/x-raw,format=I420,width=1920,height=1200 ! v4l2h264enc ! "video/x-h264,level=(string)5.1" ! h264parse ! matroskamux ! filesink location=foo.mkv

Do please note that:

• 1920x1200 exceeds the spec of the encoder.
• The TC358743 datasheet also states it is "Up to 1080P @ 60fps", so 1920x1200@60 exceeds the spec there too.

[Maxim Devaev]

Your corruption looks to be in the H264 block column format having an incorrect column stride.

It confused me too. But it worked before without any problems and stopped working after the specified commit. I really went through all the commits to make sure that this is where everything broke down.

Sorry, I'm not going to try setting up your system.

Well, it's not my fault that the conditions are too specific :) Please check the suspicious block you are talking about. I will try to reproduce the problem on gstreamer, although it will take some time. Maybe this is the format, because I've tried UYVY and RGB24, and you're using I420 in your pipeline. Also, garbage is not generated every time: if the gop interval is 30, then the problem usually occurs near the end of the gop, and a new keyframe of a new gop restores the flow. That is, the keyframes are fine. And it doesn't happen every gop.

uStreamer has the ability to encode H264 without keyframes, and make a keyframe only by request, for example, when a new client connects. This allows us to significantly save traffic when the entire stream consists of only I-frames. So now, if I try to use this with 1920x1200, I will almost immediately get a broken stream, while on 1920x1080 it continues to work fine (before the commit, everything worked fine in both resolutions). Which leads me to think that somehow only the I-frames are broken.

Do please note that 1920x1200 exceeds the spec

I understand. I specifically did extensive testing before offering you a patch about the 5.1 profile. All these years, from batch to batch, the hardware is able to handle 1920x1200, so the problem is only in the software. At least I would like the behavior of the software not to change here.

[6by9]

Test firmware that fixes the anomaly noted in the code. I can't say for definite that it is the cause, but it could cause linked issues.

https://drive.google.com/file/d/1YJA6LEaWwOFNbI4DGoU6UZuSS9EYVpEh/view?usp=sharing

[Maxim Devaev]

Yes, this is it. Fixed on my setup. Thank you for solving this.

[Maxim Devaev]

I've checked the latest commit raspberrypi/firmware@9f24f4b and it seems the fix is not included there.

[6by9]

There was further internal review required to ensure we were actually using the right numbers in a few places.
That's been finalised, so should be merged in the next day or so.

[Maxim Devaev]

Thanks for the info. I'll test it when you do the push.

[popcornmix]

Thanks for the info. I'll test it when you do the push.

latest rpi-update firmware should have the fix.

[Maxim Devaev]

Yes, it seems everything is working now. Thanks again!