mattdesl / mp4-wasm-encoder

mp4-wasm-encoder

💡 Update - Project Moved

See the mp4-h264 module which wraps most of this demo up as a ES module.

What follows below is early R&D / proof-of-concept.

---

Fast client-side MP4 encoding demo based on a fork of Trevor Sundberg's npm library. Currently only works in Chrome due to OffscreenCanvas.

[ Demo Link ]

• Creates a H264-encoded MP4 video in the browser
• Can be used for long videos (thousands of frames)
• The WASM+JS dependency is ~200 KB before gzip (much smaller than ffmpeg.wasm)
• Uses WASM SIMD if enabled (Chrome only; first enable #enable-webassembly-simd in about:flags)
• Uses OffscreenCanvas to speed up rendering in a web worker (Chrome only)

A 5 second 1920x1080 60FPS MP4 takes about 7 seconds to encode with Chrome and SIMD enabled (may be faster/slower depending on your hardware).

How is this fast? (for a browser)

This is mostly based on Trevor Sundberg's work with h264-mp4-encoder (thanks!). Here, I've mostly just been exploring how to improve performance:

• Use workers and OffscreenCanvas for rendering
  • Save frames with transferToImageBitmap() then into a RGB buffer with WebGL
  • Convert RGB to YUV in the worker and then post that to the main encoder thread
• Uses a true WASM file, and WASM SIMD where available
• Sets Emscripten memory directly to avoid passing any array buffers to C/C++
• A few additional tweaks and new flags added to my C/C++ fork
• I'm also using an entirely different library to build the MP4 container – shrinking the final WASM+JS size from ~700KB to ~200KB. Compare to the pure-JS version of h264-mp4-encoder which is ~1.7MB.

How it could be faster?

It's still pretty slow compared to native, some ways it could be faster/cleaner:

• Use a second worker (thread) to handle encoding, this might not speed things up much but at least will take a load off main UI thread
• Ensure that WASM version of minih264 library is indeed taking advantage of SIMD (lots of NEON code that doesn't compile there)
• Open to other ideas! Please create an issue if you think you see any ways to make it faster.

How can it work on FireFox, mobile phones, etc?

The h264-mp4-encoder already works on most browsers, this demo is just to see how more advanced browser features could make it faster: OffscreenCanvas, dynamic imports in a web worker, WASM, and SIMD.

Just Give me the WASM!

In the h264 folder is a drop-in WASM+JS files for SIMD and non-SIMD supported environments. These must be imported as an ES module, see ./encoder/main.js, but otherwise you can get it running without any dependencies or build tools.

Feel free to use the WASM for your own purposes, though this repo won't be maintained like a typical library, so you might rather use the original non-WASM version at h264-mp4-encoder library which will probably receive more frequent updates.

In future, I plan to wrap some of this R&D work in a more 'official' MP4+GIF encoder library that will receive regular updates.

License

MIT, see LICENSE.md for details.