In this paper, we introduce and analyze a variant of the Thompson sampling
(TS) algorithm for contextual bandits. At each round, traditional TS requires
samples from the current posterior distribution, which is usually intractable.
To circumvent this issue, approximate inference techniques can be used and
provide samples with distribution close to the posteriors. However, current
approximate techniques yield to either poor estimation (Laplace approximation)
or can be computationally expensive (MCMC methods, Ensemble sampling...). In
this paper, we propose a new algorithm, Varational Inference Thompson sampling
VITS, based on Gaussian Variational Inference. This scheme provides powerful
posterior approximations which are easy to sample from, and is computationally
efficient, making it an ideal choice for TS. In addition, we show that VITS
achieves a sub-linear regret bound of the same order in the dimension and
number of round as traditional TS for linear contextual bandit. Finally, we
demonstrate experimentally the effectiveness of VITS on both synthetic and real
world datasets.

该论文介绍和分析了一种上下文赌博问题的变体的汤普森采样（TS）算法，提出了一种基于高斯变分推理的新算法 VITS，并通过实验展示了其在合成和真实世界数据集上的有效性。

VITS：上下文推测中的变分推断汤姆逊采样

VITS : Variational Inference Thomson Sampling for contextual bandits

In order to integrate uncertainty estimates into deep time-series modelling,
Kalman Filters (KFs) (Kalman et al., 1960) have been integrated with deep
learning models, however, such approaches typically rely on approximate
inference techniques such as variational inference which makes learning more
complex and often less scalable due to approximation errors. We propose a new
deep approach to Kalman filtering which can be learned directly in an
end-to-end manner using backpropagation without additional approximations. Our
approach uses a high-dimensional factorized latent state representation for
which the Kalman updates simplify to scalar operations and thus avoids hard to
backpropagate, computationally heavy and potentially unstable matrix
inversions. Moreover, we use locally linear dynamic models to efficiently
propagate the latent state to the next time step. The resulting network
architecture, which we call Recurrent Kalman Network (RKN), can be used for any
time-series data, similar to a LSTM (Hochreiter & Schmidhuber, 1997) but uses
an explicit representation of uncertainty. As shown by our experiments, the RKN
obtains much more accurate uncertainty estimates than an LSTM or Gated
Recurrent Units (GRUs) (Cho et al., 2014) while also showing a slightly
improved prediction performance and outperforms various recent generative
models on an image imputation task.

本文提出了一种名为 RKN 的循环卡尔曼网络，它可以在不使用其他近似方法（如变分推断）的情况下，使用反向传播直接学习高维因式分解潜在状态表示，从而具有能够精确估计不确定性的能力，并在图像插值任务中优于多种生成模型。