Fail to replicate frame extraction process on ActivityNet-Entities

[Liu Hualin]

Hi Luowei,

Thanks for sharing your code on frame-wise feature extraction. I am currently using this code to extract frame-wise features for ActivityNet-Entities dataset. I have a problem with the frame sampling. When I use this code base to sample the frames, I found out the number of frames I extracted in some videos are different from the number of frames you extracted. Sometimes I extract more frames than you sometimes I extract less frames than you. I wonder what could be the reasons. Could you please help me with it? Thank you!

By the way, here is the setting from my side.
opencv2: 3.4.1
python: 3.6.9
ffmpeg: 4.2.2

[Luowei Zhou]

@liuhualin333 Please refer to these two issues:
salesforce/densecap#14
salesforce/densecap#25 (comment)

[Liu Hualin]

Hi Luowei,

I have referred to these issues you mentioned. It does solve some of the inconsistencies I have. However, I still have nearly 5000+ videos with inconsistent number of sampled frames. I found out that for most of the time, my sampled frames has just one more frame than the row count in feature file. I wonder if you have any solution to this or maybe point out what could be the possible reasons. Thank you!

One of the example is as follows:

video_name: UD4RnQp5tPY
total_frames: 6603
duration: 220
frame_sampled(my script): 441
row_num in rgb_motion_1d: 440

This accounts for nearly 3900+ inconsistencies.

Another main problem is different number of sampled frames:

Here is the example:

video_name: S47vfJ9g3Mw
total_frames: 5543
total_frames in anet_duration_frame: 5606

[Luowei Zhou]

@liuhualin333 Sorry for the delay. This might result from the discrepancy in videos. Please make sure you have downloaded the videos using the official crawl. Also, some videos are altered by the owner over time which could result in the major difference in number of sampled frames. For example, could you check the duration of video S47vfJ9g3Mw is identical to the duration info in my meta file?

Also, we were using an older version of FFmpeg back in 2018 which is a potential reason for the +/-1 frame issue.

[Liu Hualin]

Hi Luowei,

Thanks for your reply. I used the official crawl script to download the video. And I just check the duration of my downloaded video S47vfJ9g3Mw has the same duration info as in your meta file anet_duration_frame. The duration is 187.5 seconds.

About the FFmpeg version, I wonder if you still be able to recall the version number you were using. Thanks!