Imitation learning is a widely used policy learning method that enables intelligent agents to acquire complex skills from expert demonstrations. The input to the imitation learning algorithm is usually composed of both the current observation and historical observations since the most recent observation might not contain enough information. This is especially the case with image observations, where a single image only includes one view of the scene, and it suffers from a lack of motion information and object occlusions. In theory, providing multiple observations to the imitation learning agent will lead to better performance. However, surprisingly people find that sometimes imitation from observation histories performs worse than imitation from the most recent observation. In this paper, we explain this phenomenon from the information flow within the neural network perspective. We also propose a novel imitation learning neural network architecture that does not suffer from this issue by design. Furthermore, our method scales to high-dimensional image observations. Finally, we benchmark our approach on two widely used simulators, CARLA and MuJoCo, and it successfully alleviates the copycat problem and surpasses the existing solutions.

本文研究探讨了模仿学习中的信息流，并提出了一种新的神经网络结构，解决了图像信号缺乏运动信息和物体遮挡的问题，并通过在CARLA和MuJoCo两个模拟器上的基准测试，成功地缓解了模仿过程中的拷贝问题并超越了现有的解决方案。

通过残差动作预测解决视觉模仿学习中的Copycat问题