In collaborative human-robot manipulation, a robot must predict human intents and adapt its actions accordingly to smoothly execute tasks. However, the human's intent in turn depends on actions the robot takes, creating a chicken-or-egg problem. Prior methods ignore such inter-dependency and instead train marginal intent prediction models independent of robot actions. This is because training conditional models is hard given a lack of paired human-robot interaction datasets. Can we instead leverage large-scale human-human interaction data that is more easily accessible? Our key insight is to exploit a correspondence between human and robot actions that enables transfer learning from human-human to human-robot data. We propose a novel architecture, InteRACT, that pre-trains a conditional intent prediction model on large human-human datasets and fine-tunes on a small human-robot dataset. We evaluate on a set of real-world collaborative human-robot manipulation tasks and show that our conditional model improves over various marginal baselines. We also introduce new techniques to tele-operate a 7-DoF robot arm and collect a diverse range of human-robot collaborative manipulation data, which we open-source.

在协作人机操纵中，机器人必须预测人类意图并相应地调整动作以顺利执行任务。本研究通过利用人机动作之间的对应关系，提出了一种新颖的架构 InteRACT，通过在大规模人机数据集上预训练条件意图预测模型并在小规模人机数据集上微调，评估了一组真实世界的协作人机操纵任务，并证明了我们的条件模型优于各种边缘基线模型。还引入了一些新技术来远程操作一个具有七自由度的机器人臂并收集了各种各样的人机协作操纵数据，该数据已开源。

InteRACT: 基于机器人行动的人类意图预测的Transformer模型