About num_frames for UniformTemporalSubsample

[Enrico Randellini]

Hi,

for a task of video classification I have a dataset with video clips ranging in length from 1 to 5 seconds. I also know that the frame rate is fixed at 10 fps for each video clip. I set num_frames = 32 for UniformTemporalSubsample(num_frames) transformation. This transformation takes care of equally sampling a number of frames equal to num_frames. Is it correct? Only the sampled frames will be used by the model to perform the train and inference steps. Is it correct? What happens for those videos clips that have fewer frames than num_frames? I tried to run the train and it didn't stop even though num_frames is greater than the number of frames contained in those videos with a duration of less than 3 seconds.

Thanks

[desi]

Equally sampling num_frames?

Yes

Only sampled frames will be used?

Yes

what happens if video clips have fewer samples than num_frames

Frames will be repeated. You will always get num_frames frames back (or else you won't be able to put frames in a batch).

You can see the implementation here (it's 3 lines) and test the behaviour with your specific numbers.