Try to add alpha channel support to the player by myself, but get an alpha buffer array which values are all zero from wasm

[LoS-Light]

I want to play a webm video(VP8) include alpha channel,
so I try to add alpha channel support to the player by myself, then I dig into the source code.

~

First of all, I check the "vpx_image_t" struct definition from "vpx_image.h", so I can know that "planes[3]" and "stride[3]" are for alpha channel.
...
#define VPX_PLANE_ALPHA 3 /< A (Transparency) plane */
unsigned char *planes[4]; /< pointer to the top left pixel for each plane */
int stride[4]; /**< stride between rows for each plane */

~

Next, in "ogv-decoder-video-vpx.c", I modify some functions below
static vpx_image_t* copy_image(vpx_image_t src) {
...
...
vpx_image_t dest = vpx_img_alloc(NULL, src->fmt, src->d_w, src->d_h, 16);
copy_plane(dest, src, 0, width, height);
copy_plane(dest, src, 1, chromaWidth, chromaHeight);
copy_plane(dest, src, 2, chromaWidth, chromaHeight);
copy_plane(dest, src, 3, width, height); //<---------- I add it
return dest;
}
...
...
static int process_frame_return(void *user_data) {
...
...
ogvjs_callback_frame(image->planes[0], image->stride[0],
image->planes[1], image->stride[1],
image->planes[2], image->stride[2],
image->planes[3], image->stride[3], //<---------- I add it
image->w, height,
chromaWidth, chromaHeight,
image->d_w, image->d_h, // crop size
0, 0, // crop pos
image->r_w, image->r_h); // render size
...
...
}

~

Next, modify the declaration from "ogv-decoder-video.h"
extern void ogvjs_callback_frame(unsigned char *bufferY, int strideY,
unsigned char *bufferCb, int strideCb,
unsigned char *bufferCr, int strideCr,
unsigned char *bufferA, int strideA, //<---------- I add it
int width, int height,
int chromaWidth, int chromaHeight,
int picWidth, int picHeight,
int picX, int picY,
int displayWidth, int displayHeight);

~

Next, go to the "ogv-decoder-video-callbacks.js" to modify the function "ogvjs_callback_frame"
ogvjs_callback_frame: function(bufferY, strideY,
bufferCb, strideCb,
bufferCr, strideCr,
bufferA, strideA, //<---------- I add it
width, height,
chromaWidth, chromaHeight,
picWidth, picHeight,
picX, picY,
displayWidth, displayHeight) {
...
...
var recycled = Module['recycledFrames'],
frame,
lenY = height * strideY,
lenCb = chromaHeight * strideCb,
lenCr = chromaHeight * strideCr,
lenA = height * strideA; //<---------- I add it
...
...
next['y']['bytes'].length === lenY &&
next['u']['bytes'].length === lenCb &&
next['v']['bytes'].length === lenCr &&
next['a']['bytes'].length === lenA //<---------- I add it
...
...
'v': {
'bytes': new Uint8Array(lenCr),
'stride': strideCr
},
'a': { //<---------- I add this part
'bytes': new Uint8Array(lenA),
'stride': strideA
}
...
...
copyAndTrim(frame['y']['bytes'], bufferY, strideY, height, picX, picY, picWidth, picHeight, 0);
copyAndTrim(frame['u']['bytes'], bufferCb, strideCb, chromaHeight, chromaPicX, chromaPicY, chromaPicWidth, chromaPicHeight, 128);
copyAndTrim(frame['v']['bytes'], bufferCr, strideCr, chromaHeight, chromaPicX, chromaPicY, chromaPicWidth, chromaPicHeight, 128);
copyAndTrim(frame['a']['bytes'], bufferA, strideA, height, picX, picY, picWidth, picHeight, 0); //<---------- I add it

// And queue up the output buffer!
Module['frameBuffer'] = frame;

console.log(frame); //<---------- I add it to log the frame
}

~

After buiding, I play a test video(vp8 webm with alpha channel)(720x1280), and check the frame log from the chrome console, and in the 'a' property, Uint8Array length is 1024000 but the values are all zero.
a: {bytes: Uint8Array(1024000), stride: 800}

~

I can't figure out where I am wrong, so I create this new issue to ask for help.
Thank you very much.

[Brooke Vibber]

I don't know whether that technique works for getting at the alpha channel or not, but you should be aware that yuv-canvas does not support an alpha channel for the YUV to RGB conversion at this time.

You'd have to modify yuv-canvas to add this support.

[Brooke Vibber]

Closing in favor of older entry #590