Video Operator should process video of any length and size
aatmanvaidya opened this issue · comments
A core component of Feluda are Operators , they are modules that help us analyse media items such a text, image, video, audio etc. One key operator is the video operator which takes in a video file as input and generates the vector representation of the file in output.
Link to the Video Vec Operator - https://github.com/tattle-made/feluda/blob/main/src/core/operators/vid_vec_rep_resnet.py
Currently, The video operator has a size limit of the videos it can process. The task is to modify the operator so that it can process large video files. When we say video of any length and size should be processed, what we mean is that the RAM usage should be constant and not increase linearly with video size/length
Expected Outcome
- Refactor the video operator to support large files.
- Profile the modified operator for CPU and RAM usage against a variety of large video files (15 mins to 1 hour). Some methods on profiling have been documented on the Wiki here.
- Write a unit test for handling of large video files.
Files to look at
- Video Operator - feluda/src/core/operators/vid_vec_rep_resnet.py at main · tattle-made/feluda
- Video Operator Test file - feluda/src/core/operators/test_vid_vec_rep_resnet.p
Mind if I have a look at this?
@kes-edge Its not on our immediate roadmap, so we won't be looking at it. Feel free to work on it and share any insights/progress. Good luck