Model-Based Reinforcement Learning (MBRL) offers a promising direction for sample efficient learning, often achieving state of the art results for continuous control tasks. However, many existing MBRL methods rely on combining greedy policies with exploration heuristics, and even those which utilize principled exploration bonuses construct dual objectives in an ad hoc fashion. In this paper we introduce Ready Policy One (RP1), a framework that views MBRL as an active learning problem, where we aim to improve the world model in the fewest samples possible. RP1 achieves this by utilizing a hybrid objective function, which crucially adapts during optimization, allowing the algorithm to trade off reward v.s. exploration at different stages of learning. In addition, we introduce a principled mechanism to terminate sample collection once we have a rich enough trajectory batch to improve the model. We rigorously evaluate our method on a variety of continuous control tasks, and demonstrate statistically significant gains over existing approaches.

本文介绍了Ready Policy One (RP1)，将基于模型的强化学习视为一个主动学习问题，利用混合目标函数，在优化期间关键性的适应，以便在学习的不同阶段权衡奖励与探索，同时介绍了一个原则性的机制以停止样本收集。在多个连续控制任务中对方法进行了严格评估，并证明了与现有方法相比的显著增益。

Ready Policy One: 通过积极学习构建世界