In this work, we study how to build a robotic system that can solve multiple
3D manipulation tasks given language instructions. To be useful in industrial
and household domains, such a system should be capable of learning new tasks
with few demonstrations and solving them precisely. Prior works, like PerAct
and RVT, have studied this problem, however, they often struggle with tasks
requiring high precision. We study how to make them more effective, precise,
and fast. Using a combination of architectural and system-level improvements,
we propose RVT-2, a multitask 3D manipulation model that is 6X faster in
training and 2X faster in inference than its predecessor RVT. RVT-2 achieves a
new state-of-the-art on RLBench, improving the success rate from 65% to 82%.
RVT-2 is also effective in the real world, where it can learn tasks requiring
high precision, like picking up and inserting plugs, with just 10
demonstrations. Visual results, code, and trained model are provided at:
this https URL

本研究提出了 RVT-2，一种多任务 3D 操纵模型，通过构架和系统层面的改进，使其在训练时快 6 倍、推理时快 2 倍，实现了在 RLBench 上的成功率从 65％提高到 82％，能够通过仅仅 10 个演示来学习需要高精度的任务，如拾取和插入插头。

RVT-2: 从少量示范中学习精确操作

RVT-2: Learning Precise Manipulation from Few Demonstrations

Reinforcement learning holds the promise of enabling autonomous robots to
learn large repertoires of behavioral skills with minimal human intervention.
However, robotic applications of reinforcement learning often compromise the
autonomy of the learning process in favor of achieving training times that are
practical for real physical systems. This typically involves introducing
hand-engineered policy representations and human-supplied demonstrations. Deep
reinforcement learning alleviates this limitation by training general-purpose
neural network policies, but applications of direct deep reinforcement learning
algorithms have so far been restricted to simulated settings and relatively
simple tasks, due to their apparent high sample complexity. In this paper, we
demonstrate that a recent deep reinforcement learning algorithm based on
off-policy training of deep Q-functions can scale to complex 3D manipulation
tasks and can learn deep neural network policies efficiently enough to train on
real physical robots. We demonstrate that the training times can be further
reduced by parallelizing the algorithm across multiple robots which pool their
policy updates asynchronously. Our experimental evaluation shows that our
method can learn a variety of 3D manipulation skills in simulation and a
complex door opening skill on real robots without any prior demonstrations or
manually designed representations.

本文介绍一种基于深度 Q 函数算法的深度强化学习方法，能够实现在真实的物理机器人上进行复杂的 3D 操作任务学习，并通过多个机器人异步汇聚优化策略更新等技术进一步提高训练效率。