Feature request: Specify frame or percentage
phillipseamore opened this issue · comments
Thanks for these great projects, I've been using imagor for a while and imagorvideo will help me get rid of a bunch of shell scripts I'm using to do thumbnails.
I'd like to request the option of getting a specific frame or a frame close to a certain percentage of the duration.
Requesting frame number 74:
http://localhost:8000/unsafe/fit-in/300x200/filters:frame(74)/https://raw.githubusercontent.com/ietf-wg-cellar/matroska-test-files/master/test_files/test4.mkv
Requesting a frame from around the middle of a video (floating number between 0.0-1.0 would be percentage of duration, integer a frame):
http://localhost:8000/unsafe/fit-in/300x200/filters:frame(0.5)/https://raw.githubusercontent.com/ietf-wg-cellar/matroska-test-files/master/test_files/test4.mkv
0.0 would then be first frame and 1.0 would be last.
I work with a lot of files that are single keyframe MP4's. Imagorvideo doesn't seem to handle them, giving error: "imagor: 406 ffmpeg: invalid data found when processing input". Attached is one example of such a file (the image is mostly dark with some streetlights [past midnight over here]). I'm hoping that being able to request the first frame will allow me to use those as input.
single_keyframe.mp4
@phillipseamore would you test if this docker image build fixes the single keyframe MP4's issue?
docker run --rm -p 8000:8000 ghcr.io/cshum/imagorvideo@sha256:06501186b9d3cf7c193ba5b1df7855956d1ab4dea735aac13ed8aa708e17e3ea -imagor-unsafe
https://github.com/cshum/imagorvideo/pkgs/container/imagorvideo/44751825?tag=master
@cshum Just tested with a number of keyframe files and it works perfectly. Thanks for the quick turnaround!
The fix is now available at v0.1.4.
The requesting frame number frame(74) sounds reasonable enough, I am keen to implement it in coming versions.
Requesting frame percentage would be way more tricky though. The current implementation is based on selecting the first n frames then compute, without looping through the entire video. Seeking to a specific location would be an entirely different paradigm, presumably way more resource intensive.
That being said, I am very new to the ffmpeg libraries. For anyone experienced with ffmpeg I am very keen to hear your opinion. As usual, any PRs are welcomed!
Thanks again, I've updated and can now start to get rid of all those shell scripts.
Requesting specific frames (or close enough keyframe, though I think ffmpeg has improved seeking enough that it will return an actual frame instead of the closest keyframe now without much additional overhead and combined seeking isn't faster [I used to seek twice, first to approximately before the frame I needed and then to the exact frame]) is a really nice feature to have.
The percentage requests is great when the content isn't known beforehand (it's better to have a frame from the middle than one from the start if it has intro/titles etc.) and specifically if the duration isn't known. But I see that duration is returned in a /meta query ("duration":144000000000 for one of the 144 second files I tested) so from that one could calculate the frame needed (adding FPS to /meta would make that perfect). Perhaps these operations could be limited to sources from local storage to keep resource use down.
v0.2.0 added FPS to meta if frame processing is activated, also changed duration to milliseconds instead:
https://github.com/cshum/imagorvideo#metadata
Added frame(n) and max_frames(n) filters thought they are currently based on the existing "select first n frames" mechanism.
Thanks for that. I was wondering why I couldn't get past frame 42, so that explains it. I noticed you have a new branch working on seeking, it looks like you are retrieving the file separately from ffmpeg, I think that ffmpeg should have all the capabilities to perform the retrieval itself (including using byte-range to only get the part of the file that would be needed, e.g. first getting moov to get all the necessary data on the file and then calculating a window for the offset needed to seek).
The master branch build adding seek support based on time or percentage:
https://github.com/cshum/imagorvideo/pkgs/container/imagorvideo/46819985?tag=master
docker run --rm -p 8000:8000 ghcr.io/cshum/imagorvideo@sha256:45e1b8889b6a0d202da9475a8932544f52d19d36f131526c35ea500fbd854222 -imagor-unsafe
You may test out frame(2m1s) and frame(0.7) filter:
http://localhost:8000/unsafe/filters:frame(2m1s)/http://commondatastorage.googleapis.com/gtv-videos-bucket/sample/BigBuckBunny.mp4
http://localhost:8000/unsafe/filters:frame(0.7)/http://commondatastorage.googleapis.com/gtv-videos-bucket/sample/BigBuckBunny.mp4
This is working great! Thanks. To the best of my knowledge this is now the only OSS image server that supports videos with percentages for thumbnails and also the only one with RMSE.
Features now live at v0.3.3