In the field of robotic manipulation, deep imitation learning is recognized
as a promising approach for acquiring manipulation skills. Additionally,
learning from diverse robot datasets is considered a viable method to achieve
versatility and adaptability. In such research, by learning various tasks,
robots achieved generality across multiple objects. However, such multi-task
robot datasets have mainly focused on single-arm tasks that are relatively
imprecise, not addressing the fine-grained object manipulation that robots are
expected to perform in the real world. This paper introduces a dataset of
diverse object manipulations that includes dual-arm tasks and/or tasks
requiring fine manipulation. To this end, we have generated dataset with 224k
episodes (150 hours, 1,104 language instructions) which includes dual-arm fine
tasks such as bowl-moving, pencil-case opening or banana-peeling, and this data
is publicly available. Additionally, this dataset includes visual attention
signals as well as dual-action labels, a signal that separates actions into a
robust reaching trajectory and precise interaction with objects, and language
instructions to achieve robust and precise object manipulation. We applied the
dataset to our Dual-Action and Attention (DAA), a model designed for
fine-grained dual arm manipulation tasks and robust against covariate shifts.
The model was tested with over 7k total trials in real robot manipulation
tasks, demonstrating its capability in fine manipulation.

该研究介绍了一个包含双臂任务和 / 或需要细致操纵的多样化对象操作数据集，数据集包括 224k 个剧集、双臂精细任务以及语言指令，并应用于 Dual-Action and Attention (DAA) 模型，该模型在真实机器人操作任务中展示了其在细致操纵方面的能力。

双臂精细操纵的多任务机器人数据

Multi-task robot data for dual-arm fine manipulation

Deep imitation learning is a promising method for dexterous robot
manipulation because it only requires demonstration samples for learning
manipulation skills. In this paper, deep imitation learning is applied to tasks
that require force feedback, such as bottle opening. However, simple visual
feedback systems, such as teleoperation, cannot be applied because they do not
provide force feedback to operators. Bilateral teleoperation has been used for
demonstration with force feedback; however, this requires an expensive and
complex bilateral robot system. In this paper, a new master-to-robot (M2R)
transfer learning system is presented that does not require robots but can
still teach dexterous force feedback-based manipulation tasks to robots. The
human directly demonstrates a task using a low-cost controller that resembles
the kinematic parameters of the robot arm. Using this controller, the operator
can naturally feel the force feedback without any expensive bilateral system.
Furthermore, the M2R transfer system can overcome domain gaps between the
master and robot using the gaze-based imitation learning framework and a simple
calibration method. To demonstrate this, the proposed system was evaluated on a
bottle-cap-opening task that requires force feedback only for the master
demonstration.

本文提出了一种新的主控器到机器人 (M2R) 迁移学习系统，通过低成本控制器和基于注视的模仿学习框架及简单的校准方法，使操作者可以自然地感受到力反馈而不需要昂贵的双边系统，进而克服主控器和机器人之间的域差异，使得机器人也能够完成需要力反馈的灵巧操作任务，作者在瓶盖开启任务上进行了验证。

无需机器人的机器人训练：基于深度模仿学习的主控策略传递

Training Robots without Robots: Deep Imitation Learning for Master-to-Robot Policy Transfer

Deep imitation learning is a promising approach that does not require
hard-coded control rules in autonomous robot manipulation. The current
applications of deep imitation learning to robot manipulation have been limited
to reactive control based on the states at the current time step. However,
future robots will also be required to solve tasks utilizing their memory
obtained by experience in complicated environments (e.g., when the robot is
asked to find a previously used object on a shelf). In such a situation, simple
deep imitation learning may fail because of distractions caused by complicated
environments. We propose that gaze prediction from sequential visual input
enables the robot to perform a manipulation task that requires memory. The
proposed algorithm uses a Transformer-based self-attention architecture for the
gaze estimation based on sequential data to implement memory. The proposed
method was evaluated with a real robot multi-object manipulation task that
requires memory of the previous states.

本文提出了一种使用基于 Transformer 的自注意力结构实现眼神预测，以实现具有记忆的机器人操作任务的算法。通过将机器人视觉输入的连续序列作为输入，该算法能够通过使用传统的深度模仿学习方法并结合眼动预测来实现多物体操作任务。实验结果表明，该方法成功地改善了机器人在具有记忆要求的复杂环境中的操作表现。