very serious problem with k600 test set
yusq45 opened this issue · comments
After downloading, there are a lot of videos in the training set, and the actual number of correct videos is only more than 3,000
Like -BC1iA8OOoQ_000006_000016.mp4, it should be in the training set, but it is in the test set you provided
Hi.
There is a large overlap between the kinetics700 training set and the
kinetics600 test set.
From the kinetics700 paper (https://arxiv.org/pdf/1907.06987v2.pdf):
*It is therefore largely safe to use models that have been trained on
Kinetics-600 to evaluate the Kinetics-700 test set (the activity-net
evaluation website explicitly ignores the predictions on those 3% clips
when evaluating on the test set). It is however not safe to train on
Kinetics-700 and then evaluate on Kinetics-600 – many of the
validation/test clips from Kinetics-600 are in the training set of
Kinetics-700 !*
Best,
Joao
…On Tue, Jul 25, 2023 at 7:24 AM yusq45 ***@***.***> wrote:
Like -BC1iA8OOoQ_000006_000016.mp4, it should be in the training set, but
it is in the test set you provided
—
Reply to this email directly, view it on GitHub
<#41 (comment)>,
or unsubscribe
<https://github.com/notifications/unsubscribe-auth/ADXKU2U72JVUSO2CGDLXQQ3XR5RCVANCNFSM6AAAAAA2WRXBMQ>
.
You are receiving this because you are subscribed to this thread.Message
ID: ***@***.***>
Thank you so much