The ability to plan at many different levels of abstraction enables agents to
envision the long-term repercussions of their decisions and thus enables
sample-efficient learning. This becomes particularly beneficial in complex
environments from high-dimensional state space such as pixels, where the goal
is distant and the reward sparse. We introduce Forecaster, a deep hierarchical
reinforcement learning approach which plans over high-level goals leveraging a
temporally abstract world model. Forecaster learns an abstract model of its
environment by modelling the transitions dynamics at an abstract level and
training a world model on such transition. It then uses this world model to
choose optimal high-level goals through a tree-search planning procedure. It
additionally trains a low-level policy that learns to reach those goals. Our
method not only captures building world models with longer horizons, but also,
planning with such models in downstream tasks. We empirically demonstrate
Forecaster's potential in both single-task learning and generalization to new
tasks in the AntMaze domain.

Forecaster 是一种深度分层强化学习方法，通过对抽象层次的状态转换动态建模和训练世界模型，在高维状态空间（如像素）等复杂环境中规划高层次目标，并通过树搜索规划程序选择最优高层次目标，从而捕捉建立具有较长时间跨度的世界模型和在下游任务中规划使用这些模型的潜力。在 AntMaze 领域中的单任务学习和新任务泛化方面，我们通过实验证明了 Forecaster 的潜力。