Generalization has been one of the major challenges for learning dynamics models in model-based reinforcement learning. However, previous work on action-conditioned dynamics prediction focuses on learning the pixel-level motion and thus does not generalize well to novel environments with different object layouts. In this paper, we present a novel object-oriented framework, called object-oriented dynamics predictor (OODP), which decomposes the environment into objects and predicts the dynamics of objects conditioned on both actions and object-to-object relations. It is an end-to-end neural network and can be trained in an unsupervised manner. To enable the generalization ability of dynamics learning, we design a novel CNN-based relation mechanism that is class-specific (rather than object-specific) and exploits the locality principle. Empirical results show that OODP significantly outperforms previous methods in terms of generalization over novel environments with various object layouts. OODP is able to learn from very few environments and accurately predict dynamics in a large number of unseen environments. In addition, OODP learns semantically and visually interpretable dynamics models.

本文提出了一种名为OODP的目标导向动态预测框架，通过将环境分解为对象并预测对象间的动态来提高学习动态模型的泛化性能，该框架使用端到端神经网络实现，并可进行无监督学习。实验结果表明，OODP在各种不同对象布局的新环境中具有显著的泛化能力，且能够从极少量的环境中学习并可以准确地预测大量未见过的环境的动态，同时它还能学习具有语义和视觉可解释的动态模型。

面向对象的动态预测器