AmenRa / ranx

⚡️A Blazing-Fast Python Library for Ranking Evaluation, Comparison, and Fusion 🐍

Home Page:https://amenra.github.io/ranx

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

[BUG] dcg and dcg_burges do not work in the compare function

mpetri opened this issue · comments

Describe the bug

when adding the newly available dcg or dcg_burges metric in the compare function I get this error:

report = compare(
    qrels=qrels,
    runs=runs,
    metrics=["recall@10","ndcg","rbp.90","rbp.50","dcg_burges"],
    max_p=0.05,   # P-value threshold
    stat_test='student'
)
Traceback (most recent call last):
  File "/local/home/mkp/data/gap2kic/eval/./run.py", line 32, in <module>
    print(report)
  File "/home/mkp/.asdf/installs/python/3.10.9/lib/python3.10/site-packages/ranx/data_structures/report.py", line 338, in __str__
    return self.to_table()
  File "/home/mkp/.asdf/installs/python/3.10.9/lib/python3.10/site-packages/ranx/data_structures/report.py", line 143, in to_table
    label = self.get_metric_label(x)
  File "/home/mkp/.asdf/installs/python/3.10.9/lib/python3.10/site-packages/ranx/data_structures/report.py", line 122, in get_metric_label
    return f"{metric_labels[m]}"
KeyError: 'dcg_burges'

however in the same file when I do

res = evaluate(qrels, run, ["recall@10","ndcg","rbp.90","rbp.50","dcg","dcg_burges"])

everything works as intended.

Yep, I forget about them. Thanks for pointing it out!
Fixed in v0.3.14