https://w3c.github.io/webcodecs/#dom-videoframe-videoframe-data-init should validate the plane layout according the coded width and height

Question

https://w3c.github.io/webcodecs/#dom-videoframe-videoframe-data-init should validate the plane layout according the coded width and height

youennf opened this issue a year ago · comments

https://w3c.github.io/webcodecs/#dom-videoframe-videoframe-data-init step 9 is computing the plane layout according parsedRect which is computed from visibleWidth/visibleHeight.
It seems codedWidth and codedHeight should be used instead, which would be consistent with checking whether the buffer is large enough.
See also https://bugs.webkit.org/show_bug.cgi?id=256848.

youennf · Answer 1 · Fri Jul 07 2023 17:08:36 GMT+0800 (China Standard Time)

Switching to using codedWidth/codedHeight seems to align with what Chrome does AIUI: https://source.chromium.org/chromium/chromium/src/+/main:third_party/blink/renderer/modules/webcodecs/video_frame.cc;l=849;drc=a3636d1ffd79769d79cb736dcf0066eeef408762

Dan Sanders · Answer 2 · Tue Jul 11 2023 02:12:06 GMT+0800 (China Standard Time)

Agreed, combinedLayout is only used to run the check in step 11, which should be making sure that the buffer is large enough to contain the entire coded region of the frame. It's not necessary to compute anything related to the visible region here, but Compute Layout and Allocation Size algorithm is shared with copyTo() which does.

The complexity comes from having to support explicit and implicit layouts. The actual check just needs to ensure that the buffer contains the full planeCodedHeight * planeStride bytes for every plane.

It could be worth making this more obvious because using planeCodedWidth * planeSampleBytes for the last row instead of planeStride might be logical but would differ from the spec intention.