Imitation learning is a promising paradigm for training robot control
policies, but these policies can suffer from distribution shift, where the
conditions at evaluation time differ from those in the training data. A popular
approach for increasing policy robustness to distribution shift is interactive
imitation learning (i.e., DAgger and variants), where a human operator provides
corrective interventions during policy rollouts. However, collecting a
sufficient amount of interventions to cover the distribution of policy mistakes
can be burdensome for human operators. We propose IntervenGen (I-Gen), a novel
data generation system that can autonomously produce a large set of corrective
interventions with rich coverage of the state space from a small number of
human interventions. We apply I-Gen to 4 simulated environments and 1 physical
environment with object pose estimation error and show that it can increase
policy robustness by up to 39x with only 10 human interventions. Videos and
more results are available at this https URL

通过 IntervenGen (I-Gen) 数据生成系统，只需借助少数人工干预，便能自主生成大量具有状态空间丰富覆盖的纠正干预数据，从而将策略的鲁棒性提高多达 39 倍。

IntervenGen：强化和高效机器人模仿学习的干预数据生成

IntervenGen: Interventional Data Generation for Robust and  Data-Efficient Robot Imitation Learning

Corrective interventions while a robot is learning to automate a task provide
an intuitive method for a human supervisor to assist the robot and convey
information about desired behavior. However, these interventions can impose
significant burden on a human supervisor, as each intervention interrupts other
work the human is doing, incurs latency with each context switch between
supervisor and autonomous control, and requires time to perform. We present
LazyDAgger, which extends the interactive imitation learning (IL) algorithm
SafeDAgger to reduce context switches between supervisor and autonomous
control. We find that LazyDAgger improves the performance and robustness of the
learned policy during both learning and execution while limiting burden on the
supervisor. Simulation experiments suggest that LazyDAgger can reduce context
switches by an average of 60% over SafeDAgger on 3 continuous control tasks
while maintaining state-of-the-art policy performance. In physical fabric
manipulation experiments with an ABB YuMi robot, LazyDAgger reduces context
switches by 60% while achieving a 60% higher success rate than SafeDAgger at
execution time.

LazyDAgger 是一种互动式的模仿学习算法，通过减少监督员和自主控制之间的上下文切换次数，提高了学习和执行过程中所学策略的性能和鲁棒性，且在限制监督员负担的同时，能够在 3 个连续控制任务中平均减少 60％的上下文切换，并在 ABB YuMi 机器人的物理织物操作实验中成功率比 SafeDAgger 高 60％。