Humans are able to seamlessly visually imitate others, by inferring their
intentions and using past experience to achieve the same end goal. In other
words, we can parse complex semantic knowledge from raw video and efficiently
translate that into concrete motor control. Is it possible to give a robot this
same capability? Prior research in robot imitation learning has created agents
which can acquire diverse skills from expert human operators. However,
expanding these techniques to work with a single positive example during test
time is still an open challenge. Apart from control, the difficulty stems from
mismatches between the demonstrator and robot domains. For example, objects may
be placed in different locations (e.g. kitchen layouts are different in every
house). Additionally, the demonstration may come from an agent with different
morphology and physical appearance (e.g. human), so one-to-one action
correspondences are not available. This paper investigates techniques which
allow robots to partially bridge these domain gaps, using their past
experience. A neural network is trained to mimic ground truth robot actions
given context video from another agent, and must generalize to unseen task
instances when prompted with new videos during test time. We hypothesize that
our policy representations must be both context driven and dynamics aware in
order to perform these tasks. These assumptions are baked into the neural
network using the Transformers attention mechanism and a self-supervised
inverse dynamics loss. Finally, we experimentally determine that our method
accomplishes a $\sim 2$x improvement in terms of task success rate over prior
baselines in a suite of one-shot manipulation tasks.

本文介绍了一种使用神经网络和 Transformer 注意机制的方法，通过向机器人展示上下文视频来缩小机器人学习中的领域差距，并实现了对单次操作任务的 2 倍成功率提升。