Efficient exploration is an unsolved problem in Reinforcement Learning which
is usually addressed by reactively rewarding the agent for fortuitously
encountering novel situations. This paper introduces an efficient active
exploration algorithm, Model-Based Active eXploration (MAX), which uses an
ensemble of forward models to plan to observe novel events. This is carried out
by optimizing agent behaviour with respect to a measure of novelty derived from
the Bayesian perspective of exploration, which is estimated using the
disagreement between the futures predicted by the ensemble members. We show
empirically that in semi-random discrete environments where directed
exploration is critical to make progress, MAX is at least an order of magnitude
more efficient than strong baselines. MAX scales to high-dimensional continuous
environments where it builds task-agnostic models that can be used for any
downstream task.

论文提出了一种高效的 Model-Based Active eXploration (MAX) 算法用于强化学习中的有效探索，该算法使用前向模型的集合来规划观察新事件的行为，优化代理行为，通过基于贝叶斯的探索估计集合成员之间的未来预测差异衡量新颖性，实验证明在半随机离散环境中 MAX 比强基线算法至少高效一个数量级，能够扩展到高维连续环境。