Robot Imitation Learning (IL) is a widely used method for training robots to
perform manipulation tasks that involve mimicking human demonstrations to
acquire skills. However, its practicality has been limited due to its
requirement that users be trained in operating real robot arms to provide
demonstrations. This paper presents an innovative solution: an Augmented
Reality (AR)-assisted framework for demonstration collection, empowering
non-roboticist users to produce demonstrations for robot IL using devices like
the HoloLens 2. Our framework facilitates scalable and diverse demonstration
collection for real-world tasks. We validate our approach with experiments on
three classical robotics tasks: reach, push, and pick-and-place. The real robot
performs each task successfully while replaying demonstrations collected via
AR.

通过增强现实技术辅助的演示获取框架，该论文提出了一种创新的解决方案，使非机器人专业人员能够使用 HoloLens 2 等设备为机器人模仿学习提供演示，从而实现了可扩展和多样化的真实世界任务演示收集。通过在三个经典机器人任务（到达、推动和拾取放置）上进行实验证明了该方法的有效性，实际机器人能够成功执行每个任务并回放通过增强现实收集的演示。

可扩展机器人模仿学习的增强现实演示

Augmented Reality Demonstrations for Scalable Robot Imitation Learning

Language provides a way to break down complex concepts into digestible
pieces. Recent works in robot imitation learning use language-conditioned
policies that predict actions given visual observations and the high-level task
specified in language. These methods leverage the structure of natural language
to share data between semantically similar tasks (e.g., "pick coke can" and
"pick an apple") in multi-task datasets. However, as tasks become more
semantically diverse (e.g., "pick coke can" and "pour cup"), sharing data
between tasks becomes harder, so learning to map high-level tasks to actions
requires much more demonstration data. To bridge tasks and actions, our insight
is to teach the robot the language of actions, describing low-level motions
with more fine-grained phrases like "move arm forward". Predicting these
language motions as an intermediate step between tasks and actions forces the
policy to learn the shared structure of low-level motions across seemingly
disparate tasks. Furthermore, a policy that is conditioned on language motions
can easily be corrected during execution through human-specified language
motions. This enables a new paradigm for flexible policies that can learn from
human intervention in language. Our method RT-H builds an action hierarchy
using language motions: it first learns to predict language motions, and
conditioned on this and the high-level task, it predicts actions, using visual
context at all stages. We show that RT-H leverages this language-action
hierarchy to learn policies that are more robust and flexible by effectively
tapping into multi-task datasets. We show that these policies not only allow
for responding to language interventions, but can also learn from such
interventions and outperform methods that learn from teleoperated
interventions. Our website and videos are found at
this https URL

通过使用语言动作，桥接任务和动作，实现从人类介入中学习的灵活策略，并展示了这些策略在多任务数据集上的鲁棒性和灵活性。

RT-H: 使用语言的动作层次结构

RT-H: Action Hierarchies Using Language

Humans are able to seamlessly visually imitate others, by inferring their
intentions and using past experience to achieve the same end goal. In other
words, we can parse complex semantic knowledge from raw video and efficiently
translate that into concrete motor control. Is it possible to give a robot this
same capability? Prior research in robot imitation learning has created agents
which can acquire diverse skills from expert human operators. However,
expanding these techniques to work with a single positive example during test
time is still an open challenge. Apart from control, the difficulty stems from
mismatches between the demonstrator and robot domains. For example, objects may
be placed in different locations (e.g. kitchen layouts are different in every
house). Additionally, the demonstration may come from an agent with different
morphology and physical appearance (e.g. human), so one-to-one action
correspondences are not available. This paper investigates techniques which
allow robots to partially bridge these domain gaps, using their past
experience. A neural network is trained to mimic ground truth robot actions
given context video from another agent, and must generalize to unseen task
instances when prompted with new videos during test time. We hypothesize that
our policy representations must be both context driven and dynamics aware in
order to perform these tasks. These assumptions are baked into the neural
network using the Transformers attention mechanism and a self-supervised
inverse dynamics loss. Finally, we experimentally determine that our method
accomplishes a $\sim 2$x improvement in terms of task success rate over prior
baselines in a suite of one-shot manipulation tasks.

本文介绍了一种使用神经网络和 Transformer 注意机制的方法，通过向机器人展示上下文视频来缩小机器人学习中的领域差距，并实现了对单次操作任务的 2 倍成功率提升。

一次即视视觉模仿的变形金刚

Transformers for One-Shot Visual Imitation

An option is a short-term skill consisting of a control policy for a
specified region of the state space, and a termination condition recognizing
leaving that region. In prior work, we proposed an algorithm called Deep
Discovery of Options (DDO) to discover options to accelerate reinforcement
learning in Atari games. This paper studies an extension to robot imitation
learning, called Discovery of Deep Continuous Options (DDCO), where low-level
continuous control skills parametrized by deep neural networks are learned from
demonstrations. We extend DDO with: (1) a hybrid categorical-continuous
distribution model to parametrize high-level policies that can invoke discrete
options as well continuous control actions, and (2) a cross-validation method
that relaxes DDO's requirement that users specify the number of options to be
discovered. We evaluate DDCO in simulation of a 3-link robot in the vertical
plane pushing a block with friction and gravity, and in two physical
experiments on the da Vinci surgical robot, needle insertion where a needle is
grasped and inserted into a silicone tissue phantom, and needle bin picking
where needles and pins are grasped from a pile and categorized into bins. In
the 3-link arm simulation, results suggest that DDCO can take 3x fewer
demonstrations to achieve the same reward compared to a baseline imitation
learning approach. In the needle insertion task, DDCO was successful 8/10 times
compared to the next most accurate imitation learning baseline 6/10. In the
surgical bin picking task, the learned policy successfully grasps a single
object in 66 out of 99 attempted grasps, and in all but one case successfully
recovered from failed grasps by retrying a second time.

本文介绍了 Discovery of Deep Continuous Options （DDCO）算法，这是一种针对机器人模仿学习的扩展型深度发现选项（DDO）算法，通过学习从示例中获得的深度神经网络参数化的低级连续控制技能。我们通过混合范畴 - 连续分布模型来扩展 DDO，以参数化可以调用离散选项以及连续控制动作的高级策略，并使用交叉验证方法减少 DDO 需要使用的选项数量指定，通过实验验证了 DDCO 算法的有效性。