[Feature Request] Add interpolated recall-precision plot function
EingeoelterBolt opened this issue · comments
Is your feature request related to a problem? Please describe.
First of all: This is a really nice library! It helps a lot!
My request is regarding a recall-precision graphic. When I read TREC-related papers, they very often used the interpolated precision-recall-plot to visualize the performance of IR-Systems which are being compared to each other. They also use the graphs to understand which IR-System yields a higher recall value, shows a higher precision value, and generally has a better performance.
Describe the solution you'd like
Since I love using this library, it would be great, if there were a function for generating such a precision-recall-plot or just an example/notebook for generating this plot using the ranx library in the documentation.
Describe alternatives you've considered
I've already considered creating these plots myself with the functions that the library offers. However, it seems quite complex as the interpolation greatly complicates this work for me.
Additional context
Here is an example interpolated recall-precision plot:
More information can be found on:
https://nlp.stanford.edu/IR-book/html/htmledition/evaluation-of-ranked-retrieval-results-1.html
and
https://trec.nist.gov/pubs/trec15/appendices/CE.MEASURES06.pdf (p.4)
Hi, thanks for the very detailed suggestion/request!
I think this would be a nice addition to the library!
I will take a look to the linked documents in the coming days.
I found a really nice lecture on youtube, which explains the recall-precision plot really nice. It can be found here:
https://youtu.be/x0SkRpIVMUs?t=1066
I also found out, that the official trec_eval program (https://github.com/usnistgov/trec_eval) returns the values for creating this plot. My workaround is actually, that i create runs using ranx, export the runs to a .trec-file and use the official trec_eval program
Hi, sorry for the delay.
I finally have a working implementation for computing the graph!
I still have to figure out a couple of usability things.
You can expect to be able to compute the Precision-Recall graph with ranx soon.