[Bug] The case in Figure 11 is not in the MMBench?
Richar-Du opened this issue · comments
Prerequisite
- I have searched Issues and Discussions but cannot get the expected help.
- The bug has not been fixed in the latest version.
Type
I'm evaluating with the officially supported tasks/models/datasets.
Environment
The same as the official
Reproduces the problem - code/configuration sample
question_file = 'mmbench_dev_20230712.tsv'
question_df = pd.read_csv(question_file, sep='\t', encoding='utf-8')
physical_relation = question_df[question_df['category']=='physical_relation']
Reproduces the problem - command or script
NO
Reproduces the problem - error message
I iterate the physical_relation and did not find the data displayed in Figure 11.
Other information
No
@Richar-Du , the two cases do exist in MMBench, but in the test split. That's why you can not find them in the train split.