We consider the Imitation Learning (IL) setup where expert data are not collected on the actual deployment environment but on a different version. To address the resulting distribution shift, we combine behavior cloning (BC) with a planner that is tasked to bring the agent back to states visited by the expert whenever the agent deviates from the demonstration distribution. The resulting algorithm, POIR, can be trained offline, and leverages online interactions to efficiently fine-tune its planner to improve performance over time. We test POIR on a variety of human-generated manipulation demonstrations in a realistic robotic manipulation simulator and show robustness of the learned policy to different initial state distributions and noisy dynamics.

本文介绍一种与专家数据收集环境不同的Imitation Learning（模仿学习）方法，该方法结合行为克隆和计划策略，通过将代理人带回专家访问的状态解决数据分布漂移问题，以提高性能。作者将这一算法命名为POIR，并在实际的机器人操作模拟器中进行了多种实验测试，并表明该学习策略对于不同的初始状态分布和嘈杂的动态具有很好的鲁棒性。

Robust Imitation by Return-to-Distribution Planning