While reinforcement learning (RL) has made great advances in scalability, exploration and partial observability are still active research topics. In contrast, Bayesian RL (BRL) provides a principled answer to both state estimation and the exploration-exploitation trade-off, but struggles to scale. To tackle this challenge, BRL frameworks with various prior assumptions have been proposed, with varied success. This work presents a representation-agnostic formulation of BRL under partially observability, unifying the previous models under one theoretical umbrella. To demonstrate its practical significance we also propose a novel derivation, Bayes-Adaptive Deep Dropout rl (BADDr), based on dropout networks. Under this parameterization, in contrast to previous work, the belief over the state and dynamics is a more scalable inference problem. We choose actions through Monte-Carlo tree search and empirically show that our method is competitive with state-of-the-art BRL methods on small domains while being able to solve much larger ones.

本文提出了一种表示无关的、针对部分可观测情况下的贝叶斯强化学习的理论框架，并提出了一种基于dropout网络的新方法BADDr，旨在解决BRL方法在拓展性上存在的瓶颈，并证实其在处理规模较大的情况时的有效性。

BADDr: 基于贝叶斯适应性的深度Dropout RL用于POMDPs