johnmyleswhite / BanditsBook

Code for my book on Multi-Armed Bandit Algorithms

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

Epsilon Greedy: graph for Accuracy, Performance...sensitive to the declaration order of the arm ?

phsimon opened this issue · comments

By coding for retrieving performance curve as shown in chapter "analyzing Results from Monte Carlo (chapter 4) study" Approach 1 (Proba of selecting best arm) Approach 2 (Average Reward), I noticed that changing the order of the arm, may change dramatically the curves.
For instance if I declare the mean of my five arms as follow :
means=[0.8, 0.9, 0.1, 0.5, 0.5] n_arms=len(means) random.shuffle(means) arms=[BernoulliArm(mu) for mu in means]

The shuffle change the order within means list, therefore the order of the arms, and the performance curve may be very different.

for instance, considering [0.9, 0.5, 0.1, 0.5, 0.8] as order after shuffling I get this curve (average reward per time):

image

whereas considering [0.1, 0.5, 0.9, 0.5, 0.8] , I get

image

Do you have an explanation ? Same phenomenon for proba of selecting best arm (but no so much).

Parameters:
num_sims=1000
horizon=250
epsilon=0.1