Testing accuracy of KQAPro
prerna-agarwal-iitd opened this issue · comments
Hi @leezythu. Thanks for your interesting work. I was particularly interested in the KQAPro results. Since KQAPro haven't released the ground truth answers of the test set, the numbers reported in the paper are on validation set or on test set only?
Thank you for your attention to our work. For KQA Pro, we handed over the prediction results on test set to the authors of KQA Pro for evaluation. Please refer to http://thukeg.gitee.io/kqa-pro/leaderboard.html. In experiments, the accuracy of the model on the validation set and the test set is almost the same.
@leezythu thank you for your response. Do you have the results file on validation set or the category wise accuracy numbers on validation set? If you could provide that, it would be very helpful.
No problem, you can give me your email and I will send you the file.
My email id is: prerna.agarwal@scai.iitd.ac.in. Thanks a lot!
@leezythu I am eagerly waiting for the results file on validation set or the category wise accuracy numbers on validation set. Thanks!
I remember I have sent it, can you check the mailbox？
hi @leezythu I think you sent it on prema.agarwal@scai.iitd.ac.in but my email Id is prerna.agarwal@scai.iitd.ac.in. I think there is a typo between "M" and "RN" in my name :)
could you please re-send it to the correct email address?
The spelling has no mistakes. Maybe there are some other mistakes. I have uploaded the files to the IR_results folder.
predict_val_egst_ir.txt (Accuracy:45.50) corresponds to the model with EGST and IR.
predict_val_egst.txt (Accuracy:33.16) corresponds to the model with EGST.
If you have any further questions, feel free to contact me.