Learning dynamics models accurately is an important goal for Model-Based
Reinforcement Learning (MBRL), but most MBRL methods learn a dense dynamics
model which is vulnerable to spurious correlations and therefore generalizes
poorly to unseen states. In this paper, we introduce Causal Dynamics Learning
for Task-Independent State Abstraction (CDL), which first learns a
theoretically proved causal dynamics model that removes unnecessary
dependencies between state variables and the action, thus generalizing well to
unseen states. A state abstraction can then be derived from the learned
dynamics, which not only improves sample efficiency but also applies to a wider
range of tasks than existing state abstraction methods. Evaluated on two
simulated environments and downstream tasks, both the dynamics model and
policies learned by the proposed method generalize well to unseen states and
the derived state abstraction improves sample efficiency compared to learning
without it.

本文介绍了一种名为 CDL 的任务无关状态抽象的因果关系动力学学习模型，它不仅从学习的动力学中产生状态抽象，而且还通过消除状态变量和动作之间的不必要依赖来提高泛化能力，并在两个模拟环境和下游任务中表现出比现有状态抽象方法更好的样本效率和对未知状态更好的泛化性能。