GrindGods / PyXAB

Python implementation of X-armed bandit algorithms

PyXAB - Python X-Armed Bandit

(Primary Version) Python implementation of different algorithms for X-armed bandit problems, also known as continuous-arm bandit (CAB), Lipschitz bandit, global optimization (GO), bandit-based blackbox optimization.

These algorithms rely on the hierarchical partitioning of the parameter space X. (Currently our code only supports continuous, connected domain, but the algorithms are designed for any measurable space)

Quick Example

First define the blackbox objective, the parameter domain, the partition of the space, and the algorithm, e.g.

target = Garland()
domain = [[0, 1]]
partition = BinaryPartition
algo = T_HOO(rounds=1000, domain=domain, partition=partition)

At every round t, call algo.pull(t) to get a point. After receiving the (stochastic) reward for the point, call algo.receive_reward(t, reward) to give the algorithm the feedback

point = algo.pull(t)
reward = target.f(point) + np.random.uniform(-0.1, 0.1) # Uniform noise example
algo.receive_reward(t, reward)

More detailed examples are provided here

Citation

If you use our package in your research or projects, we kindly ask you to cite our work

@article{li2021optimum,
  title={Optimum-statistical Collaboration Towards General and Efficient Black-box Optimization},
  author={Li, Wenjie and Wang, Chi-Hua, Qifan Song and Cheng, Guang},
  journal={arXiv preprint arXiv:2106.09215},
  year={2021}
}

Implemented Features:

(1). X-armed bandit algorithms

• Algorithm starred are meta-algorithms (wrappers)

Algorithm	Research Paper	Year
T-HOO	X-Armed Bandit	2011
HCT	Online Stochastic Optimization Under Correlated Bandit Feedback	2014
POO*	Black-box optimization of noisy functions with unknown smoothness	2015
GPO*	General Parallel Optimization Without A Metric	2019
PCT	General Parallel Optimization Without A Metric	2019
VHCT	Optimum-statistical Collaboration Towards General and Efficient Black-box Optimization	2021
VPCT	N.A. (GPO + VHCT)	N.A.

(2). Partition of the parameter space

Partition	Description
BinaryPartition	Equal-size binary partition of the parameter space, the split dimension is chosen uniform randomly
RandomBinaryPartition	The same as BinaryPartition but with a randomly chosen split point
DimensionBinaryPartition	Equal-size partition of the space with a binary split on each dimension, the number of children of one node is 2^d

(3). Synthetic Objectives

• Some of these objectives can be found here

Objectives	Mathematical Description	Image
Garland
DoubleSine
DifficultFunc
Ackley
Himmelblau
Rastrigin