[BUG] dcg and dcg_burges do not work in the compare function
mpetri opened this issue · comments
Matthias Petri commented
Describe the bug
when adding the newly available dcg
or dcg_burges
metric in the compare function I get this error:
report = compare(
qrels=qrels,
runs=runs,
metrics=["recall@10","ndcg","rbp.90","rbp.50","dcg_burges"],
max_p=0.05, # P-value threshold
stat_test='student'
)
Traceback (most recent call last):
File "/local/home/mkp/data/gap2kic/eval/./run.py", line 32, in <module>
print(report)
File "/home/mkp/.asdf/installs/python/3.10.9/lib/python3.10/site-packages/ranx/data_structures/report.py", line 338, in __str__
return self.to_table()
File "/home/mkp/.asdf/installs/python/3.10.9/lib/python3.10/site-packages/ranx/data_structures/report.py", line 143, in to_table
label = self.get_metric_label(x)
File "/home/mkp/.asdf/installs/python/3.10.9/lib/python3.10/site-packages/ranx/data_structures/report.py", line 122, in get_metric_label
return f"{metric_labels[m]}"
KeyError: 'dcg_burges'
however in the same file when I do
res = evaluate(qrels, run, ["recall@10","ndcg","rbp.90","rbp.50","dcg","dcg_burges"])
everything works as intended.
Matthias Petri commented
I think metric labels are missing here: https://github.com/AmenRa/ranx/blob/cf7b00c829a223c988c080ab39562637527346ca/ranx/data_structures/report.py#L24C38-L24C38
Elias Bassani commented
Yep, I forget about them. Thanks for pointing it out!
Fixed in v0.3.14