How can I get the test accuracy of action benchmark after the competition ends ？
Hou9612 opened this issue · comments
#20
Hello，I have a new question. The competition EPIC-KITCHENS-100 Action Recognition will end at June 1st，so how can I get the test accuracy of action benchmark after the competition ends？(just for getting the test accuracy of my model).
Notice that:
*) The leaderboard for EK100 will open again beginning of July, after winners are announced. You can continue to evaluate on the test set throughout the year - only closed to assess the annual winners.
*) The leaderboard for EK55 is always open of submission an will remain so.
You should evaluate your method on both the validation and the test sets. The test set is evaluated through the servers.
@dimadamen Thanks for your replies in three issues(#20, #22, #27 in epic-kitchens-55-annotations). I have some additional questions：
(1) Is there have any submission limit(e.g. one day can just submit once) in the open testing phase?
(2) In the challenge, can I use both training and validation dataset to train my model?
(3) In the challenge, can I use other public datasets（e.g. EGTEA, Kinetics, Ego4D）to train my model?
@dmoltisanti Also thanks for your relpy.
Yes, daily limit applies in open testing phase. This avoids overfitting to the test set.
Yes, you can use train+val in training your model. This is what we used to report results in the paper for baselines.
Yes, please read the FAQ in https://epic-kitchens.github.io/2022#challenges carefully where we answered this Q. The answer extends for other datasets such as Kinetics.
--
Q. Can I use the large-scale Ego4D dataset in the EPIC-KITCHENS challenges?
A. Yes, you can. You need to explicitly declare that in the Supervision Level Scales (SLS) of your submission as follows.
When using Ego4D for pretraining, set the Pretraining (PT) flag to level 3 when pretraining in self-supervised manner, and 4 when pretraining with supervision.
When using Ego4D labels as training data, set the Training Data (TD) flag to 4 if using Ego4D solely andd 5 if using Ego4D with other private datasets