Replace the queue with AWS Lambda?
veltman opened this issue · comments
Rendering a video on demand might be a good Lambda use case, need to investigate this further.
Potential upsides: No Redis, no queue, much more modular, much easier installation for the server/editor alone
Potential downsides: AWS lock-in, AWS documentation nightmares, debugging nightmares, long delays for long videos, hard limit of 300s rendering time
This might help: https://github.com/bbc/node-canvas-lambda-deps
I know popup uses a lambda for their similar tool, and I think the TAL shortcut tool does as well, fwiw.
The 300s (5min) time limit for the function can now be extended to 15min.
You could potentially distribute the rendering of the video in segments across concurrent Lambdas and then concat the video segments with ffmpeg to speed the process up (and potentially get rid of the fifteen-minute limit).
Let's just spin up an AWS Fargate instance run it till the rendering is done and kill it. It's almost similar to lambda, gets no limit on execution time, and can handle load and complexity.