paperswithcode / sota-extractor

The SOTA extractor pipeline

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

Year not found in evaluation-tables.json

dhamaris opened this issue · comments

Hi, I just noticed that in the case of https://paperswithcode.com/sota/visual-question-answering-on-gqa-test2019

The agent lxmert-adv-txt appears twice (that is correct from the JSON) but the year is 2020 (there is no date in evaluation-tables.json, see paper_date as null):

 {
              "code_links": [],
              "metrics": {
                "Accuracy": "61.12",
                "Binary": "78.07",
                "Consistency": "91.13",
                "Distribution": "5.55",
                "Open": "46.16",
                "Plausibility": "84.8",
                "Validity": "96.36"
              },
              "model_links": [],
              "model_name": "lxmert-adv-txt",
              "paper_date": null,
              "paper_title": "",
              "paper_url": "",
              "uses_additional_data": false
            },
            {
              "code_links": [],
              "metrics": {
                "Accuracy": "61.1",
                "Binary": "77.99",
                "Consistency": "91.08",
                "Distribution": "5.52",
                "Open": "46.19",
                "Plausibility": "84.82",
                "Validity": "96.36"
              },
              "model_links": [],
              "model_name": "lxmert-adv-txt",
              "paper_date": null,
              "paper_title": "",
              "paper_url": "",
              "uses_additional_data": false
            }

Where do you get the year from?
Thank you

Please refer to my response in #18

Ok, so the data does not only come from evaluation-tables.json, but from sources like https://eval.ai/web/challenges/challenge-page/225/leaderboard/733#leaderboardrank-18

Thank you, I close this issue