Confidence intervals are a crucial building block in the analysis of various
online learning problems. The analysis of kernel based bandit and reinforcement
learning problems utilize confidence intervals applicable to the elements of a
reproducing kernel Hilbert space (RKHS). However, the existing confidence
bounds do not appear to be tight, resulting in suboptimal regret bounds. In
fact, the existing regret bounds for several kernelized bandit algorithms
(e.g., GP-UCB, GP-TS, and their variants) may fail to even be sublinear. It is
unclear whether the suboptimal regret bound is a fundamental shortcoming of
these algorithms or an artifact of the proof, and the main challenge seems to
stem from the online (sequential) nature of the observation points. We
formalize the question of online confidence intervals in the RKHS setting and
overview the existing results.

研究了基于核的赌博机和强化学习问题，利用可再生核希尔伯特空间 (RKHS) 元素的置信区间，发现现有置信区间似乎不紧，导致次优的遗憾度界限，存在几种核化赌博机算法 (例如 GP-UCB，GP-TS 及其变种) 的现有遗憾度界限可能甚至不能达到亚线性，需要在在线观察点的连续性质的条件下重新界定 RKHS 环境下的在线置信区间问题并简述现有进展。