YNAT (Where is test data?)
conan1024hao opened this issue · comments
Hi, I am a Chinese student studying NLP in Japan and I am also learning Korean. I really love your KLUE website, it's much more beautiful and easy-to-understand than CLUE or JGLUE.
Description
I downloaded data and baseline code from https://klue-benchmark.com/tasks/66/data/download but I found that there is no test data so I could not run the inference script to generate submission file. And there is only inference script without train script, do you have one?
@songys @SungjoonPark @DSKSD @inmoonlight @monologg @warnikchow @myeonghwa-lee Please look at this issue if you have time, thank you.
@conan1024hao We deliberately hide the test set so that the trained model is not easily overfitted on it. You can check the performance on the test set via KLUE leaderboard. Thank you!
@inmoonlight Thank your for reply. Do you mean we need to upload our model file and inference code to use the leaderboard? Since we have no ynat-v1.1_test.json, we could not run inference.py locally.
@conan1024hao Exactly. You can find the example code at https://klue-benchmark.com/tasks/66/data/download
@inmoonlight Sorry to bother you again. I tried upload my model and inference code(which were tested OK locally) as a tar.gz but got submission fail. Since we do not know your computing environment, python version and else, it's really hard to provide codes which can run smoothly. Maybe we will just use your train and dev data for evaluation...