Giters

jind11 / MMM-MCQA

Source code for our "MMM" paper at AAAI 2020

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

How to reproduce your roberta-large baseline on DREAM ?

oklen opened this issue · comments

commented

hi,I have run your code with:
python run_classifier_roberta_exe.py dream roberta-large-mnli 1 24 and
python run_classifier_roberta_exe.py dream roberta-large-mnli 1 64
Both give acc on test about 82.0 which is lower than 85.0 that you mention in your paper. Could you share how do you get your result? Thanks very much.

commented

Hi, although effective batch size is more important, but if the gradient steps is too large, the performance may be degraded. I used the gradient steps between 4 and 6 and per gpu batch size of 2-4.