Bimanual manipulation is challenging due to precise spatial and temporal
coordination required between two arms. While there exist several real-world
bimanual systems, there is a lack of simulated benchmarks with a large task
diversity for systematically studying bimanual capabilities across a wide range
of tabletop tasks. This paper addresses the gap by extending RLBench to
bimanual manipulation. We open-source our code and benchmark comprising 13 new
tasks with 23 unique task variations, each requiring a high degree of
coordination and adaptability. To kickstart the benchmark, we extended several
state-of-the art methods to bimanual manipulation and also present a
language-conditioned behavioral cloning agent -- PerAct2, which enables the
learning and execution of bimanual 6-DoF manipulation tasks. Our novel network
architecture efficiently integrates language processing with action prediction,
allowing robots to understand and perform complex bimanual tasks in response to
user-specified goals. Project website with code is available at:
this http URL

扩展 RLBench 以进行双手操作的模拟基准测试，提供 13 项新任务和 23 个独特的任务变体，其中每个任务要求高度的协调和适应性。通过引入多种先进方法和 PerAct2 语言条件行为克隆代理，实现语言处理与动作预测的高效集成，使机器人能够理解和执行用户指定目标的复杂双手任务。

PerAct2: 一个面向双手操作任务的感知器执行框架

PerAct2: A Perceiver Actor Framework for Bimanual Manipulation Tasks

We present a challenging new benchmark and learning-environment for robot
learning: RLBench. The benchmark features 100 completely unique, hand-designed
tasks ranging in difficulty, from simple target reaching and door opening, to
longer multi-stage tasks, such as opening an oven and placing a tray in it. We
provide an array of both proprioceptive observations and visual observations,
which include rgb, depth, and segmentation masks from an over-the-shoulder
stereo camera and an eye-in-hand monocular camera. Uniquely, each task comes
with an infinite supply of demos through the use of motion planners operating
on a series of waypoints given during task creation time; enabling an exciting
flurry of demonstration-based learning. RLBench has been designed with
scalability in mind; new tasks, along with their motion-planned demos, can be
easily created and then verified by a series of tools, allowing users to submit
their own tasks to the RLBench task repository. This large-scale benchmark aims
to accelerate progress in a number of vision-guided manipulation research
areas, including: reinforcement learning, imitation learning, multi-task
learning, geometric computer vision, and in particular, few-shot learning. With
the benchmark's breadth of tasks and demonstrations, we propose the first
large-scale few-shot challenge in robotics. We hope that the scale and
diversity of RLBench offers unparalleled research opportunities in the robot
learning community and beyond.

RLBench 是一个机器人学习的挑战性基准和学习环境，旨在加速包括强化学习、模仿学习、几何计算机视觉和少样本学习在内的一些视觉导向操作研究领域的进展。