Questions about MostPopular Recall on Taobao

Question

Questions about MostPopular Recall on Taobao

UesugiErii opened this issue 3 years ago · comments

I implemented the MostPopular method myself and tested it on your dataset.
On Amazon book, I have the same result as you.
On Taobao, I have the same HitRate result as you, but the result of Recall is different from yours.
My Taobao Recall result is 0.366@20, 0.667@50
Can you open source your MostPopular code?

outside-BUPT · Answer 1 · Mon Sep 20 2021 22:17:26 GMT+0800 (China Standard Time)

I also encountered the same situation! Could the owner open source MostPopular code?

Codazzy · Answer 2 · Thu Sep 23 2021 14:58:32 GMT+0800 (China Standard Time)

Hi @UesugiErii @outside-BUPT ,

Sorry for the late reply. I add the MostPopular code in the src/ directory. Please check whether it reproduces the reported results in the paper.

R.S · Answer 3 · Fri Sep 24 2021 20:07:32 GMT+0800 (China Standard Time)

Hi, @cenyk1230 , first thank you for your reply.

In your mostpop code, ndcg and recall is different from train.py(evaluate_full), below I just take recall as an example.

Suppose the popular item is [1,2,3], only one test user, and the test user's test set is [2,2] (item_list[int(len(item_list) * 0.8):])

If use the calculation method of evaluate_full in train.py, then recall will be 0.5
If use the calculation method of mostpop.py, then recall will be 1

The possible reason for no problem on Amazon is that there are no duplicate items in the test set of the test user, but there are duplicates on Taobao.