Renumics / spotlight

Interactively explore unstructured datasets from your dataframe.

Home Page:https://renumics.com

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

Support windows in video lens

druzsan opened this issue · comments

Is your feature request related to a problem? Please describe.
Since we support window data type and allow windows to be projected onto audio, it would be great to spread window usage to other matching data types, in particular, the video.

Describe the solution you'd like
One can enhance the existing video lens in order to take optional window column and project it onto the showed videos or build a new lens from scratch (in this case, it should be able to show video with and without an optional window).

Additional context
For desired behavior, s. Audio Lens (example below).

Windows are considered to be in seconds.

A simple dataset can be build in the root of our repository using the following script. It includes not only videos and windows with many edge cases (s. comments), but also audios to showcase the existing behavior.

import pandas as pd
from renumics import spotlight

if __name__ == "__main__":
    df = pd.DataFrame(
        {
            "comment": [
                "window",
                "window until the end",
                "window from the start the end",
                "mismatching window",
                "no window",
            ],
            "audio": [
                "data/audio/8.wav",
                "data/audio/mono/gs-16b-1c-44100hz.mp3",
                "data/audio/8.wav",
                "data/audio/mono/gs-16b-1c-44100hz.mp3",
                "data/audio/8.wav",
            ],
            "video": [
                "data/videos/sea-360p.mp4",
                "data/videos/sea-360p-10s.mp4",
                "data/videos/sea-360p.mp4",
                "data/videos/sea-360p-10s.mp4",
                "data/videos/sea-360p.mp4",
            ],
            "window": [
                [0.1, 0.3],
                [5, float("NaN")],
                [float("NaN"), 0.3],
                [20, 25],
                None,
            ],
        }
    )
    spotlight.show(df)