Language provides a way to break down complex concepts into digestible
pieces. Recent works in robot imitation learning use language-conditioned
policies that predict actions given visual observations and the high-level task
specified in language. These methods leverage the structure of natural language
to share data between semantically similar tasks (e.g., "pick coke can" and
"pick an apple") in multi-task datasets. However, as tasks become more
semantically diverse (e.g., "pick coke can" and "pour cup"), sharing data
between tasks becomes harder, so learning to map high-level tasks to actions
requires much more demonstration data. To bridge tasks and actions, our insight
is to teach the robot the language of actions, describing low-level motions
with more fine-grained phrases like "move arm forward". Predicting these
language motions as an intermediate step between tasks and actions forces the
policy to learn the shared structure of low-level motions across seemingly
disparate tasks. Furthermore, a policy that is conditioned on language motions
can easily be corrected during execution through human-specified language
motions. This enables a new paradigm for flexible policies that can learn from
human intervention in language. Our method RT-H builds an action hierarchy
using language motions: it first learns to predict language motions, and
conditioned on this and the high-level task, it predicts actions, using visual
context at all stages. We show that RT-H leverages this language-action
hierarchy to learn policies that are more robust and flexible by effectively
tapping into multi-task datasets. We show that these policies not only allow
for responding to language interventions, but can also learn from such
interventions and outperform methods that learn from teleoperated
interventions. Our website and videos are found at
this https URL

通过使用语言动作，桥接任务和动作，实现从人类介入中学习的灵活策略，并展示了这些策略在多任务数据集上的鲁棒性和灵活性。

RT-H: 使用语言的动作层次结构

RT-H: Action Hierarchies Using Language

Robot learning holds the promise of learning policies that generalize
broadly. However, such generalization requires sufficiently diverse datasets of
the task of interest, which can be prohibitively expensive to collect. In other
fields, such as computer vision, it is common to utilize shared, reusable
datasets, such as ImageNet, to overcome this challenge, but this has proven
difficult in robotics. In this paper, we ask: what would it take to enable
practical data reuse in robotics for end-to-end skill learning? We hypothesize
that the key is to use datasets with multiple tasks and multiple domains, such
that a new user that wants to train their robot to perform a new task in a new
domain can include this dataset in their training process and benefit from
cross-task and cross-domain generalization. To evaluate this hypothesis, we
collect a large multi-domain and multi-task dataset, with 7,200 demonstrations
constituting 71 tasks across 10 environments, and empirically study how this
data can improve the learning of new tasks in new environments. We find that
jointly training with the proposed dataset and 50 demonstrations of a
never-before-seen task in a new domain on average leads to a 2x improvement in
success rate compared to using target domain data alone. We also find that data
for only a few tasks in a new domain can bridge the domain gap and make it
possible for a robot to perform a variety of prior tasks that were only seen in
other domains. These results suggest that reusing diverse multi-task and
multi-domain datasets, including our open-source dataset, may pave the way for
broader robot generalization, eliminating the need to re-collect data for each
new robot learning project.

本文旨在探索在机器人技能学习中实现实用的数据重用，通过使用包含多个任务和多个领域的多样化数据集进行训练，研究表明这种方法可以大大提高机器人在未曾见过的领域和任务中的成功率，大大降低了对采集昂贵的新数据集的需求。