Giters

kudkudak / common-sense-prediction

Common sense prediction using DL.

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

Try to figure out why their model works well on extrinsic eval

kudkudak opened this issue · comments

commented

We talked with Dima:

  • Evaluation, if it has negative samples, ultimately sucks, so we need extrinsic evaluation
  • Paper also figured it out, it has extrinsic evaluation, and model performs well on it. Weird!

Task is to examine hypotheses around why it happens

commented

Seems like it doesn't really work on wiki. Next steps:

  • Validate that I reproduce good results on ConceptNet tuples
  • See closest example in both wiki and ConceptNet