recommenders-team / recommenders

Best Practices on Recommendation Systems

Home Page:https://recommenders-team.github.io/recommenders/intro.html

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

[FEATURE] Add r-precision metric

daviddavo opened this issue · comments

Description

R-precision can be defined as the precision@R, with R being the number of relevant documents for the query [1]. This means that for a certain user $u$ which has $R$ relevant items, we would ask the recommendation engine for the top $R$ recommendations, and see which of them are true positives.

[1]: C. D. Manning, P. Raghavan, and H. Schütze, ‘Chapter 8. Evaluation in information retrieval’, in Introduction to information retrieval, New York: Cambridge University Press, 2008, pp. 151–175.

Expected behavior with the suggested feature

Adding R-Precision will allow evaluating systems with some characteristics.

Other comments

Implementation idea:

  1. Get the number of relevant documents (R) for all users
  2. Get the top-k recommendations of the model where $k = \max R$
  3. Calculate the R-precision for each user, and return the mean as in other metrics

@daviddavo would you be able to contribute to this?