For robots to be effectively deployed in novel environments and tasks, they
must be able to understand the feedback expressed by humans during
intervention. This can either correct undesirable behavior or indicate
additional preferences. Existing methods either require repeated episodes of
interactions or assume prior known reward features, which is data-inefficient
and can hardly transfer to new tasks. We relax these assumptions by describing
human tasks in terms of object-centric sub-tasks and interpreting physical
interventions in relation to specific objects. Our method, Object Preference
Adaptation (OPA), is composed of two key stages: 1) pre-training a base policy
to produce a wide variety of behaviors, and 2) online-updating only certain
weights in the model according to human feedback. The key to our fast, yet
simple adaptation is that general interaction dynamics between agents and
objects are fixed, and only object-specific preferences are updated. Our
adaptation occurs online, requires only one human intervention (one-shot), and
produces new behaviors never seen during training. Trained on cheap synthetic
data instead of expensive human demonstrations, our policy demonstrates
impressive adaptation to human perturbations on challenging, realistic tasks in
our user study. Videos, code, and supplementary material provided.

本文介绍了一种人机交互的方法，使用对象为中心的子任务描述人类任务，并根据特定对象检查和更新模型中的权重，以实现机器人的快速且简单的策略自适应。