Test results are different from the paper.
robotzheng opened this issue · comments
LIDM_70
Venue Match Rate : 91.9%
Task Success Rate : 74.3%
BLEU : 0.2337
Semantic Match : 63.7%
LIDM-RL_70
Venue Match Rate : 92.6%
Task Success Rate : 75.7%
BLEU : 0.2459
Semantic Match : 63.4%
LIDM-RL_100 (just set I = 100)
Venue Match Rate : 89.7%
Task Success Rate : 70.6%
BLEU : 0.2541
Semantic Match : 60.3%
not like the paper's result:
LIDM Models
LIDM, I = 50 66.9 0.238
LIDM, I = 70 61.0 0.246
LIDM, I = 100 63.2 0.242
LIDM Models + RL
LIDM, I = 50, +RL 82.4 0.231
LIDM, I = 70, +RL 81.6 0.230
LIDM, I = 100, +RL 84.6 0.240
shawnwun, thanks.