MAB and CMAB Algorithms simulations in several datasets
This repository aims at learning most popular MAB and CMAB algorithms and watch how they run. It is interesting for those wishing to start learning these topics.
Algorithms
In this version I have implemented most popular MAB and CMAB algorithms (to be completed): UCB1, EXP3, epsilon-Greedy, Thompson Sampling, LinUCB, and Linear Thompson Sampling (also known as Contextual Thompson Sampling).
Datasets
Algorithms can be simulated over several datasets that I found in Kaggle, and UCI Machine Learning: Control (artifically generated in full features and sparse features), Mushrooms, Statlog, Students Academic Performance, Recommendation System Angers Smart City (RSASM), CNAE-9, contextfree (which is contexte free CNAE-9 version). Since Covertype and Poker Hand are high volume datasets please contact me for getting them https://www.researchgate.net/profile/Nicolas_Gutowski.
Graphic drawing mode
If you want to observe accuracy convergence in real-time (for you or your students) thus you can choose the dynamic mode. Otherwise, choose static (it will run faster)
List of parameters
Algorithms: String
- linucb
- egreedy
- ucb1
- ts
- lints
- exp3
- random
Dataset: String
- control
- controlsp
- mushrooms
- statlog
- students
- rsasm
- cnae9
- contextfree
- pokerhand
- covertype
Horizon: Integer in [1;T] T=whatever you want ;-) (100,1000,1000,10000, etc.)
Graphic drawing mode : String
- static
- dynamic
How to simulate ?
For example :
-
to simulate LinUCB algorithm in Control dataset up to horizon 2100 and watch dynamic graphic drawing you need to open a terminal and enter: python execute.py control linucb 2100 dynamic
-
to simulate epsilon Greedy algorithm in Mushrooms dataset up to horizon 3000 and only watch the final graphic (not dynamically) you need to open a terminal and enter: python execute.py mushroom egreedy 3000 static
-
to simulate UCB1 algorithm in Statlog dataset up to horizon 10000 and watch dynamic graphic drawing you need to open a terminal and enter: python execute.py statlog ucb1 10000 dynamic
-
to simulate Thompson Sampling algorithm in Mushrooms dataset up to horizon 2000 and watch dynamic graphic drawing you need to open a terminal and enter: python execute.py statlog ts 10000 dynamic