Support windows in video lens
druzsan opened this issue · comments
Is your feature request related to a problem? Please describe.
Since we support window data type and allow windows to be projected onto audio, it would be great to spread window usage to other matching data types, in particular, the video.
Describe the solution you'd like
One can enhance the existing video lens in order to take optional window column and project it onto the showed videos or build a new lens from scratch (in this case, it should be able to show video with and without an optional window).
Additional context
For desired behavior, s. Audio Lens (example below).
Windows are considered to be in seconds.
A simple dataset can be build in the root of our repository using the following script. It includes not only videos and windows with many edge cases (s. comments), but also audios to showcase the existing behavior.
import pandas as pd
from renumics import spotlight
if __name__ == "__main__":
df = pd.DataFrame(
{
"comment": [
"window",
"window until the end",
"window from the start the end",
"mismatching window",
"no window",
],
"audio": [
"data/audio/8.wav",
"data/audio/mono/gs-16b-1c-44100hz.mp3",
"data/audio/8.wav",
"data/audio/mono/gs-16b-1c-44100hz.mp3",
"data/audio/8.wav",
],
"video": [
"data/videos/sea-360p.mp4",
"data/videos/sea-360p-10s.mp4",
"data/videos/sea-360p.mp4",
"data/videos/sea-360p-10s.mp4",
"data/videos/sea-360p.mp4",
],
"window": [
[0.1, 0.3],
[5, float("NaN")],
[float("NaN"), 0.3],
[20, 25],
None,
],
}
)
spotlight.show(df)