Giters

fidelity / mabwiser

[IJAIT 2021] MABWiser: Contextual Multi-Armed Bandits Library

Home Page:https://fidelity.github.io/mabwiser/

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

[Question] How to deal with cold start

risufaj opened this issue · comments

commented

Hi mabwiser team,

Thank you for this great library!

I am wondering what would be the best approach for mabwiser to be used in cold start situations, where there aren't yet any recorded decisions and rewards. Could it be possible to initialize rewards for all arms with 0? The use case I have in mind has binary rewards [0,1].

Thanks in advance!

commented

Hi @risufaj!

Thank for your interest in the library.

Great question! Cold-start is a challenging problem in many recommender system problems. Note we have a warm_start function in the MAB object that can be used to warm-start cold arms. In order to warm-start the multi-armed bandit we need a list of features for each arm that can be used to calculate pairwise similarities between the arms. See here for an example how it's used.

An obvious next question is where to get these features. In practice, one would typically have some textual information for arms, but not necessarily numerical features. Fortunately, we also developed a library called TextWiser that can be used for text featurization based on a rich set of methods. Check it out if it sounds interesting.

commented

Hi @bkleyn,

Thank you for answering!

I understand that using the function you mentioned, it would be possible to warm start an arm that is most similar to some existing arms, for which you have data. In situations where there aren't any data recorded for any of the arms, I suppose the best could be to have some sort of data collection phase, right?

commented

Yes, indeed! This is in fact an active research area, and we recently studied how to speed-up that data collection phase with publications at CPAIOR'21 and IJCAI'21.