Imitation learning in a high-dimensional environment is challenging. Most inverse reinforcement learning (IRL) methods fail to outperform the demonstrator in such a high-dimensional environment, e.g., Atari domain. To address this challenge, we propose a novel reward learning module to generate intrinsic reward signals via a generative model. Our generative method can perform better forward state transition and backward action encoding, which improves the module's dynamics modeling ability in the environment. Thus, our module provides the imitation agent both the intrinsic intention of the demonstrator and a better exploration ability, which is critical for the agent to outperform the demonstrator. Empirical results show that our method outperforms state-of-the-art IRL methods on multiple Atari games, even with one-life demonstration. Remarkably, our method achieves performance that is up to 5 times the performance of the demonstration.

通过引入一种新的奖励学习模块，可通过生成模型生成内在奖励信号。我们的生成功能可以更好地执行前向状态转换和后向动作编码，提高模块在环境中的动力学建模能力，并为模仿代理提供了模仿者的内在意图和更好的探索能力。经验证明，我们的模型在多个Atari游戏中的表现优于现有的IRL方法，即使只有一次演示，性能也是演示的5倍。

通过生成模型的内在奖励驱动的模仿学习