Imitation learning trains a policy from expert demonstrations. Imitation learning approaches have been designed from various principles, such as behavioral cloning via supervised learning, apprenticeship learning via inverse reinforcement learning, and GAIL via generative adversarial learning. In this paper, we propose a framework to analyze the theoretical property of imitation learning approaches based on discrepancy propagation analysis. Under the infinite-horizon setting, the framework leads to the value discrepancy of behavioral cloning in an order of O((1-\gamma)^{-2}). We also show that the framework leads to the value discrepancy of GAIL in an order of O((1-\gamma)^{-1}). It implies that GAIL has less compounding errors than behavioral cloning, which is also verified empirically in this paper. To the best of our knowledge, we are the first one to analyze GAIL's performance theoretically. The above results indicate that the proposed framework is a general tool to analyze imitation learning approaches. We hope our theoretical results can provide insights for future improvements in imitation learning algorithms.

本文提出了一种基于差异传播分析法来分析模仿学习方法理论特性的框架，该框架揭示了模仿学习方法在不同设置下的价值差异。通过理论与实验的结合，我们首次分析了GAIL的性能表现，并发现该方法具备比行为克隆更少的复利误差，这为模仿学习算法的未来改进提供了启示。

关于模仿学习价值偏差的研究