Planning has been very successful for control tasks with known environment
dynamics. To leverage planning in unknown environments, the agent needs to
learn the dynamics from interactions with the world. However, learning dynamics
models that are accurate enough for planning has been a long-standing
challenge, especially in image-based domains. We propose the Deep Planning
Network (PlaNet), a purely model-based agent that learns the environment
dynamics from images and chooses actions through fast online planning in latent
space. To achieve high performance, the dynamics model must accurately predict
the rewards ahead for multiple time steps. We approach this using a latent
dynamics model with both deterministic and stochastic transition components.
Moreover, we propose a multi-step variational inference objective that we name
latent overshooting. Using only pixel observations, our agent solves continuous
control tasks with contact dynamics, partial observability, and sparse rewards,
which exceed the difficulty of tasks that were previously solved by planning
with learned models. PlaNet uses substantially fewer episodes and reaches final
performance close to and sometimes higher than strong model-free algorithms.

通过图像学习环境动态、通过快速在线规划选择动作，利用深度计划网络（PlaNet）解决了连续控制任务中的挑战，同时使用的强度模型相对较少。