Usage on native linux && gstreamer-omx

Question

Usage on native linux && gstreamer-omx

rzk opened this issue 9 years ago · comments

Hello Allwinner, thanks for sharing this repo.

Could you please elaborate on topic of usage of these libraries?

they are ARM hard-float ABI libraries, so this means that you've thought about running these on native linux setup (android is armel and not very interesting in aspect of linux-sunxi)
proof:

linaro@cubieboard4:~$ readelf -h media-codec/sunxi-cedarx/LIBRARY/CODEC/VIDEO/DECODER/libvdecoder.so 
ELF Header:
  Magic:   7f 45 4c 46 01 01 01 00 00 00 00 00 00 00 00 00 
  Class:                             ELF32
  Data:                              2's complement, little endian
  Version:                           1 (current)
  OS/ABI:                            UNIX - System V
  ABI Version:                       0
  Type:                              DYN (Shared object file)
  Machine:                           ARM
  Version:                           0x1
  Entry point address:               0xf4e8
  Start of program headers:          52 (bytes into file)
  Start of section headers:          443552 (bytes into file)
  Flags:                             0x5000402, has entry point, Version5 EABI, hard-float ABI
  Size of this header:               52 (bytes)
  Size of program headers:           32 (bytes)
  Number of program headers:         6
  Size of section headers:           40 (bytes)
  Number of section headers:         29
  Section header string table index: 26
linaro@cubieboard4:~$ readelf -A media-codec/sunxi-cedarx/LIBRARY/CODEC/VIDEO/DECODER/libvdecoder.so 
Attribute Section: aeabi
File Attributes
  Tag_CPU_name: "7-A"
  Tag_CPU_arch: v7
  Tag_CPU_arch_profile: Application
  Tag_ARM_ISA_use: Yes
  Tag_THUMB_ISA_use: Thumb-2
  Tag_FP_arch: VFPv3
  Tag_Advanced_SIMD_arch: NEONv1
  Tag_ABI_PCS_wchar_t: 4
  Tag_ABI_FP_denormal: Needed
  Tag_ABI_FP_exceptions: Needed
  Tag_ABI_FP_number_model: IEEE 754
  Tag_ABI_align_needed: 8-byte
  Tag_ABI_align_preserved: 8-byte, except leaf SP
  Tag_ABI_enum_size: int
  Tag_ABI_HardFP_use: SP and DP
  Tag_ABI_VFP_args: VFP registers
  Tag_CPU_unaligned_access: v6

If these are meant to be used on native linux setup, what applications you were using to test these libraries? I was thinking about building the same setup as Raspberry Pi guys have - gstreamer-omx plugin. For more information about this, read http://freedesktop.org/wiki/GstOpenMAX/ http://gstreamer.freedesktop.org/releases/gst-omx/1.0.0.html
If you haven't thought about running gst-omx, could you please post example of usage on native linux?

Thanks again,
Best Regards.

Dmitry Beykun · Answer 1 · Mon Mar 09 2015 22:42:46 GMT+0800 (China Standard Time)

also on topic of usage:
tried VLC with this repo on Cubieboard A80

debug  : awplayer <./src/aw_omx_core.c:579>: OMXCORE API :  Get Handle b2b8e750 OMX.allwinner.video.decoder.avc b281f3e0

debug  : awplayer <omx_vdec.cpp:3272>: (f:ComponentThread, l:3272) wait for OMX_VdrvCommand_PrepareVdecLib
debug  : awplayer <omx_vdec.cpp:3769>: (f:ComponentVdrvThread, l:3769) vdrvThread receive cmd[0x0]
debug  : awplayer <omx_vdec.cpp:3774>: (f:ComponentVdrvThread, l:3774)(OMX_VdrvCommand_PrepareVdecLib)
debug  : awplayer <ve.c:157>: xxxxxxx firstMemchunk.size(0)
warning: awplayer <ve.c:632>: no bestChunk
error  : awplayer <sbm.c:70>: ESC[40;31mpSbmBuf == NULL.ESC[0m
error  : awplayer <vdecoder.c:315>: ESC[40;31mcreate stream buffer fail.ESC[0m
warning: awplayer <omx_vdec.cpp:3807>: Idle transition failed, set_vstream_info() return fail.

warning: awplayer <omx_vdec.cpp:3964>: decode fatal error[-1]
warning: awplayer <omx_vdec.cpp:3964>: decode fatal error[-1]
debug  : awplayer <omx_vdec.cpp:3274>: (f:ComponentThread, l:3274) wait for OMX_VdrvCommand_PrepareVdecLib done!

Then I have

warning: awplayer <omx_vdec.cpp:3964>: decode fatal error[-1]

repeating infinitely

What am I missing? Misconfiguration of OMX build or Cubieboard4's kernel not having something needed for cedarx? I tried switchin fbX_scaler_mode for starters, didn't help.

Dmitry Beykun · Answer 2 · Tue Mar 10 2015 02:18:02 GMT+0800 (China Standard Time)

Got it load videos on Cubieboard2 with Cubiez latest image and VLC player, but can't get it to render properly, instead of video I see green picture or picture that is wrongly aligned, colors are messed up. Unfortunately, I can't screenshot it because it seems like OpenMAX library is using disp layers and screen capture is not possible.

H264 & MPEG4 render wrongly, but get decoded. MPEG2 doesnt work. VC1 & VP8 are not sent to OpenMAX by VLC for some reason.

Here are the logs:
H264:

debug  : awplayer <./src/aw_omx_core.c:579>: OMXCORE API :  Get Handle b5049628 OMX.allwinner.video.decoder.avc b5307c68

warning: awplayer <omx_vdec.cpp:1291>: set_parameter, OMX_IndexParamPortDefinition, OutPortDef : change nBufferSize[3133440] to [3110400] to suit frame width[1920] and height[1080]
debug  : awplayer <omx_vdec.cpp:3272>: (f:ComponentThread, l:3272) wait for OMX_VdrvCommand_PrepareVdecLib
debug  : awplayer <omx_vdec.cpp:3769>: (f:ComponentVdrvThread, l:3769) vdrvThread receive cmd[0x0]
debug  : awplayer <omx_vdec.cpp:3774>: (f:ComponentVdrvThread, l:3774)(OMX_VdrvCommand_PrepareVdecLib)
debug  : awplayer <ve.c:157>: xxxxxxx firstMemchunk.size(83886080)
debug  : awplayer <omx_vdec.cpp:3274>: (f:ComponentThread, l:3274) wait for OMX_VdrvCommand_PrepareVdecLib done!
debug  : awplayer <omx_vdec.cpp:3769>: (f:ComponentVdrvThread, l:3769) vdrvThread receive cmd[0x1]
debug  : awplayer <omx_vdec.cpp:3814>: (f:ComponentVdrvThread, l:3814)(OMX_VdrvCommand_CloseVdecLib)
debug  : awplayer <omx_vdec.cpp:3130>:  stop command. pSelf->m_state[1]
debug  : awplayer <omx_vdec.cpp:3134>: (f:ComponentThread, l:3134) wait for OMX_VdrvCommand_Stop
debug  : awplayer <omx_vdec.cpp:3769>: (f:ComponentVdrvThread, l:3769) vdrvThread receive cmd[0x3]
debug  : awplayer <omx_vdec.cpp:3839>: (f:ComponentVdrvThread, l:3839)(OMX_VdrvCommand_Stop)
debug  : awplayer <omx_vdec.cpp:3136>: (f:ComponentThread, l:3136) wait for OMX_VdrvCommand_Stop done!
debug  : awplayer <omx_vdec.cpp:3873>: vdrvThread detect nStopFlag[1], exit!
debug  : awplayer <omx_vdec.cpp:3996>: ***notify: exit the componentVdrvThread!
debug  : awplayer <omx_vdec.cpp:2310>: (f:component_deinit, l:2310) two threads exit!
debug  : awplayer <omx_vdec.cpp:468>: ~omx_dec done!

MPEG4

debug  : awplayer <./src/aw_omx_core.c:579>: OMXCORE API :  Get Handle b52fe628 OMX.allwinner.video.decoder.mpeg4 1cc9690

warning: awplayer <omx_vdec.cpp:1291>: set_parameter, OMX_IndexParamPortDefinition, OutPortDef : change nBufferSize[3133440] to [3110400] to suit frame width[1920] and height[1080]
debug  : awplayer <omx_vdec.cpp:3769>: (f:ComponentVdrvThread, l:3769) vdrvThread receive cmd[0x0]
debug  : awplayer <omx_vdec.cpp:3774>: (f:ComponentVdrvThread, l:3774)(OMX_VdrvCommand_PrepareVdecLib)
debug  : awplayer <omx_vdec.cpp:3286>: (f:ComponentThread, l:3286) wait for OMX_VdrvCommand_PrepareVdecLib
debug  : awplayer <ve.c:157>: xxxxxxx firstMemchunk.size(83886080)
debug  : awplayer <omx_vdec.cpp:3288>: (f:ComponentThread, l:3288) wait for OMX_VdrvCommand_PrepareVdecLib done!
debug  : awplayer <omx_vdec.cpp:3769>: (f:ComponentVdrvThread, l:3769) vdrvThread receive cmd[0x1]
debug  : awplayer <omx_vdec.cpp:3814>: (f:ComponentVdrvThread, l:3814)(OMX_VdrvCommand_CloseVdecLib)
debug  : awplayer <omx_vdec.cpp:3130>:  stop command. pSelf->m_state[1]
debug  : awplayer <omx_vdec.cpp:3769>: (f:ComponentVdrvThread, l:3769) vdrvThread receive cmd[0x3]
debug  : awplayer <omx_vdec.cpp:3839>: (f:ComponentVdrvThread, l:3839)(OMX_VdrvCommand_Stop)
debug  : awplayer <omx_vdec.cpp:3873>: vdrvThread detect nStopFlag[1], exit!
debug  : awplayer <omx_vdec.cpp:3996>: ***notify: exit the componentVdrvThread!
debug  : awplayer <omx_vdec.cpp:3134>: (f:ComponentThread, l:3134) wait for OMX_VdrvCommand_Stop
debug  : awplayer <omx_vdec.cpp:3136>: (f:ComponentThread, l:3136) wait for OMX_VdrvCommand_Stop done!
debug  : awplayer <omx_vdec.cpp:2310>: (f:component_deinit, l:2310) two threads exit!
debug  : awplayer <omx_vdec.cpp:468>: ~omx_dec done!

MPEG2

debug  : awplayer <./src/aw_omx_core.c:579>: OMXCORE API :  Get Handle b4ec3698 OMX.allwinner.video.decoder.mpeg2 1a84790

debug  : awplayer <omx_vdec.cpp:3769>: (f:ComponentVdrvThread, l:3769) vdrvThread receive cmd[0x0]
debug  : awplayer <omx_vdec.cpp:3774>: (f:ComponentVdrvThread, l:3774)(OMX_VdrvCommand_PrepareVdecLib)
debug  : awplayer <ve.c:157>: xxxxxxx firstMemchunk.size(83886080)
debug  : awplayer <omx_vdec.cpp:3286>: (f:ComponentThread, l:3286) wait for OMX_VdrvCommand_PrepareVdecLib
debug  : awplayer <omx_vdec.cpp:3288>: (f:ComponentThread, l:3288) wait for OMX_VdrvCommand_PrepareVdecLib done!
warning: awplayer <vdecoder.c:757>: empty stream data frame submitted, pData=0xb1733000, nLength=0
warning: awplayer <vdecoder.c:757>: empty stream data frame submitted, pData=0xb1733000, nLength=0
warning: awplayer <vdecoder.c:757>: empty stream data frame submitted, pData=0xb1733000, nLength=0
warning: awplayer <vdecoder.c:757>: empty stream data frame submitted, pData=0xb1733000, nLength=0
<... this gets repeated ...>
debug  : awplayer <omx_vdec.cpp:3769>: (f:ComponentVdrvThread, l:3769) vdrvThread receive cmd[0x1]
debug  : awplayer <omx_vdec.cpp:3814>: (f:ComponentVdrvThread, l:3814)(OMX_VdrvCommand_CloseVdecLib)
debug  : awplayer <omx_vdec.cpp:3130>:  stop command. pSelf->m_state[1]
debug  : awplayer <omx_vdec.cpp:3134>: (f:ComponentThread, l:3134) wait for OMX_VdrvCommand_Stop
debug  : awplayer <omx_vdec.cpp:3769>: (f:ComponentVdrvThread, l:3769) vdrvThread receive cmd[0x3]
debug  : awplayer <omx_vdec.cpp:3839>: (f:ComponentVdrvThread, l:3839)(OMX_VdrvCommand_Stop)
debug  : awplayer <omx_vdec.cpp:3873>: vdrvThread detect nStopFlag[1], exit!
debug  : awplayer <omx_vdec.cpp:3996>: ***notify: exit the componentVdrvThread!
debug  : awplayer <omx_vdec.cpp:3136>: (f:ComponentThread, l:3136) wait for OMX_VdrvCommand_Stop done!
debug  : awplayer <omx_vdec.cpp:2310>: (f:component_deinit, l:2310) two threads exit!
debug  : awplayer <omx_vdec.cpp:468>: ~omx_dec done!

Would be nice to have some help, maybe media-codec can only work with one color space (fbX_format in fex)?

allwinner-zh · Answer 3 · Fri Mar 13 2015 12:03:34 GMT+0800 (China Standard Time)

Hi, Dmitriy Beykun,
We have updated the media-codec, the issue of the render error may be fixed, please try it again.
Thanks.

Dmitry Beykun · Answer 4 · Sat Mar 14 2015 06:07:20 GMT+0800 (China Standard Time)

Thanks.
Got it working with VLC.

Still, some documentation would be nice, something like what video decode/encode profiles you've tested on native linux setup.

Hongcai Deng · Answer 5 · Mon Mar 23 2015 23:13:16 GMT+0800 (China Standard Time)

Perfect! I would like to see more documentations!

Leo Soares · Answer 6 · Wed Apr 01 2015 01:33:28 GMT+0800 (China Standard Time)

@rzk could you get it work with gst-omx?
can you tell what have you done to get it to work with vlc?

tnks 😄

Dmitry Beykun · Answer 7 · Wed Apr 01 2015 03:33:07 GMT+0800 (China Standard Time)

@leojrfs couldnt get it to work with gst-omx. to get it to work with VLC I've used debian image and installed all bundled/generated *.so libraries to /usr/lib. VLC detects OMX by default.

Leo Soares · Answer 8 · Wed Apr 01 2015 22:10:20 GMT+0800 (China Standard Time)

@rzk thanks
What work needs to be done in order to work with gst-omx?

It would be great to have that, since there is no proper actual way to accelerate/render video on allwinner in linux. Something like rpi has on gst-omx would be OK

Dmitry Beykun · Answer 9 · Wed Apr 01 2015 22:41:22 GMT+0800 (China Standard Time)

@leojrfs I've spoke with @ebutera (he made https://github.com/ebutera/gst-plugin-cedar) and he tried to run gst-omx on OpenEmbedded system with no X11 and it failed with memory allocation errors. I couldnt get gst-omx to compile properly on my debian setup and didnt have time to investigate, unfortunatelly.

Contact @ebutera if you plan to invest some time into gst-omx for sunxi, you'll need to compile it properly on some popular system and then write configs for OMX paths. Then, hopefully, it will work for decoding.

Encoding is broken and disabled (no OMX paths, they are disabled in registry header).

Leo Soares · Answer 10 · Thu Apr 02 2015 23:50:59 GMT+0800 (China Standard Time)

@rzk i get a bunch of errors and a black video tile with vlc

all lib files are in /usr/lib, and by the output i can suppose it is loading ok.

any thoughts?

if i could get a simple h264 dump to decode with CedarX without X11 (just like the simple hello_video.c example from broadcom on rpi), i would invest time with allwinner chips and gst dev.

but i am in the process of choosing hardware for my project, and i cant even get a simple h264 dump cedarx decoder program from allwinner 😞
@allwinner-zh

Enrico Butera · Answer 11 · Fri Apr 03 2015 00:25:13 GMT+0800 (China Standard Time)

if it's a business project i can only suggest you to choose a hardware platform that works TODAY, if you think something will get fixed soon...it will not happen.
You can think "oh this is easy, just a little fix, in a couple of weeks it will get fixed/i will fix it" but the reality is that many people just wasted time and money.

Just as an example: i committed my gst plugin (based on jemk work) in Jenuary 2014, it just needs some simple fixes to be usable (same as jemk work i.e. bitrate control, qp parameters, B frames support), a year has passed and nothing happened.
I'm pretty sure that an Allwinner engineer can fix those issues in 5 minutes, avoiding all the **** with the infamous binary libraries, but here we are.

I didn't try recently but it seems that some exynos and imx6 are the best mainline supported platforms today for video encoding/decoding.

Leo Soares · Answer 12 · Fri Apr 03 2015 01:42:00 GMT+0800 (China Standard Time)

@ebutera you are absolutely right.

Thank you for the heads up and experience share!