Imitation learning has been applied to a range of robotic tasks, but can
struggle when (1) robots encounter edge cases that are not represented in the
training data (distribution shift) or (2) the human demonstrations are
heterogeneous: taking different paths around an obstacle, for instance
(multimodality). Interactive fleet learning (IFL) mitigates distribution shift
by allowing robots to access remote human teleoperators during task execution
and learn from them over time, but is not equipped to handle multimodality.
Recent work proposes Implicit Behavior Cloning (IBC), which is able to
represent multimodal demonstrations using energy-based models (EBMs). In this
work, we propose addressing both multimodality and distribution shift with
Implicit Interactive Fleet Learning (IIFL), the first extension of implicit
policies to interactive imitation learning (including the single-robot,
single-human setting). IIFL quantifies uncertainty using a novel application of
Jeffreys divergence to EBMs. While IIFL is more computationally expensive than
explicit methods, results suggest that IIFL achieves 4.5x higher return on
human effort in simulation experiments and an 80% higher success rate in a
physical block pushing task over (Explicit) IFL, IBC, and other baselines when
human supervision is heterogeneous.

本文提出了解决 Interactive Fleet Learning (分布转移) 和 Implicit Behavior Cloning (多模态) 问题的 Implicit Interactive Fleet Learning 方法，并通过模拟实验和物理实验表明其在人类辅助下具有更高的执行成功率和更高的回报。

来自不同人类监管者的隐式交互式车队学习

IIFL: Implicit Interactive Fleet Learning from Heterogeneous Human  Supervisors

Commercial and industrial deployments of robot fleets at Amazon, Nimble, Plus
One, Waymo, and Zoox query remote human teleoperators when robots are at risk
or unable to make task progress. With continual learning, interventions from
the remote pool of humans can also be used to improve the robot fleet control
policy over time. A central question is how to effectively allocate limited
human attention. Prior work addresses this in the single-robot, single-human
setting; we formalize the Interactive Fleet Learning (IFL) setting, in which
multiple robots interactively query and learn from multiple human supervisors.
We propose Return on Human Effort (ROHE) as a new metric and Fleet-DAgger, a
family of IFL algorithms. We present an open-source IFL benchmark suite of
GPU-accelerated Isaac Gym environments for standardized evaluation and
development of IFL algorithms. We compare a novel Fleet-DAgger algorithm to 4
baselines with 100 robots in simulation. We also perform a physical
block-pushing experiment with 4 ABB YuMi robot arms and 2 remote humans.
Experiments suggest that the allocation of humans to robots significantly
affects the performance of the fleet, and that the novel Fleet-DAgger algorithm
can achieve up to 8.8x higher ROHE than baselines. See
this https URL for supplemental material.

研究了交互式机群学习 (IFL) 中有限人力资源的有效分配问题，提出了一种新的度量 ROHE 以及一种 IFL 算法家族 Fleet-DAgger，并通过实验验证了该算法可以在人力资源有限的情况下实现更高的 ROHE。