While reinforcement learning (RL) methods that learn an internal model of the
environment have the potential to be more sample efficient than their
model-free counterparts, learning to model raw observations from high
dimensional sensors can be challenging. Prior work has addressed this challenge
by learning low-dimensional representation of observations through auxiliary
objectives, such as reconstruction or value prediction. However, the alignment
between these auxiliary objectives and the RL objective is often unclear. In
this work, we propose a single objective which jointly optimizes a latent-space
model and policy to achieve high returns while remaining self-consistent. This
objective is a lower bound on expected returns. Unlike prior bounds for
model-based RL on policy exploration or model guarantees, our bound is directly
on the overall RL objective. We demonstrate that the resulting algorithm
matches or improves the sample-efficiency of the best prior model-based and
model-free RL methods. While sample efficient methods typically are
computationally demanding, our method attains the performance of SAC in about
50% less wall-clock time.

本研究提出了一种单一目标的方法，该方法同时优化隐空间模型和策略以实现高回报并保持自一致性，从而在提高样本效率的同时实现更好的强化学习效果。