We consider the stochastic bandit problem with a continuous set of arms, with the expected reward function over the arms assumed to be fixed but unknown. We provide two new Gaussian process-based algorithms for continuous bandit optimization-Improved GP-UCB (IGP-UCB) and GP-Thomson sampling (GP-TS), and derive corresponding regret bounds. Specifically, the bounds hold when the expected reward function belongs to the reproducing kernel Hilbert space (RKHS) that naturally corresponds to a Gaussian process kernel used as input by the algorithms. Along the way, we derive a new self-normalized concentration inequality for vector- valued martingales of arbitrary, possibly infinite, dimension. Finally, experimental evaluation and comparisons to existing algorithms on synthetic and real-world environments are carried out that highlight the favorable gains of the proposed strategies in many cases.

本文提出了两种基于高斯过程的算法-改进的GP-UCB（IGP-UCB）和GP-Thomson采样（GP-TS），并给出了相应的遗憾边界，在连续的臂集上解决了随机赌徒问题。当期望奖励函数属于复制核希尔伯特空间（RKHS）时，边界成立。在实验评估和对合成和真实世界环境中现有算法的比较中，突出了所提出策略的优势。

核化多臂赌博机