For 3D object manipulation, methods that build an explicit 3D representation
perform better than those relying only on camera images. But using explicit 3D
representations like voxels comes at large computing cost, adversely affecting
scalability. In this work, we propose RVT, a multi-view transformer for 3D
manipulation that is both scalable and accurate. Some key features of RVT are
an attention mechanism to aggregate information across views and re-rendering
of the camera input from virtual views around the robot workspace. In
simulations, we find that a single RVT model works well across 18 RLBench tasks
with 249 task variations, achieving 26% higher relative success than the
existing state-of-the-art method (PerAct). It also trains 36X faster than
PerAct for achieving the same performance and achieves 2.3X the inference speed
of PerAct. Further, RVT can perform a variety of manipulation tasks in the real
world with just a few ($\sim$10) demonstrations per task. Visual results, code,
and trained model are provided at this https URL

本研究提出 RVT，一种基于多视角变形器的三维物体操作方法，在模拟和实际环境中均表现出色，相对于现有方法，其训练速度快 36 倍，推理速度快 2.3 倍，并仅需每项任务约 10 次演示即可达到良好的效果。

RVT: 用于 3D 对象操作的机器人视图变换器

RVT: Robotic View Transformer for 3D Object Manipulation

Coarse-to-fine Q-attention enables sample-efficient robot manipulation by
discretizing the translation space in a coarse-to-fine manner, where the
resolution gradually increases at each layer in the hierarchy. Although
effective, Q-attention suffers from "coarse ambiguity" - when voxelization is
significantly coarse, it is not feasible to distinguish similar-looking objects
without first inspecting at a finer resolution. To combat this, we propose to
envision Q-attention as a tree that can be expanded and used to accumulate
value estimates across the top-k voxels at each Q-attention depth. When our
extension, Q-attention with Tree Expansion (QTE), replaces standard Q-attention
in the Attention-driven Robot Manipulation (ARM) system, we are able to
accomplish a larger set of tasks; especially on those that suffer from "coarse
ambiguity". In addition to evaluating our approach across 12 RLBench tasks, we
also show that the improved performance is visible in a real-world task
involving small objects.

该研究提出了 Q-attention with Tree Expansion（QTE）来替代标准的 Q-attention，从而解决在机器人操作中由于分辨率较低而导致的对象类似度较高的问题。经过多项 RLBench 任务以及真实物体实验，证明我们的方法显著提高了机器人操作的效率。

树状扩展粗细 Q-attention

Coarse-to-fine Q-attention with Tree Expansion

We propose Learned Path Ranking (LPR), a method that accepts an end-effector
goal pose, and learns to rank a set of goal-reaching paths generated from an
array of path generating methods, including: path planning, Bezier curve
sampling, and a learned policy. The core idea being that each of the path
generation modules will be useful in different tasks, or at different stages in
a task. When LPR is added as an extension to C2F-ARM, our new system,
C2F-ARM+LPR, retains the sample efficiency of its predecessor, while also being
able to accomplish a larger set of tasks; in particular, tasks that require
very specific motions (e.g. opening toilet seat) that need to be inferred from
both demonstrations and exploration data. In addition to benchmarking our
approach across 16 RLBench tasks, we also learn real-world tasks, tabula rasa,
in 10-15 minutes, with only 3 demonstrations.

提出了 Learned Path Ranking 方法，通过对路径生成方法（包括路径规划、贝塞尔曲线采样和学习策略）的排序学习，可以在保持以前样本效率的同时，实现更多的任务，并在 16 个 RLBench 任务中进行了基准测试。

带有学习路径排名的粗到细的 Q-attention

Coarse-to-Fine Q-attention with Learned Path Ranking

Despite the success of reinforcement learning methods, they have yet to have
their breakthrough moment when applied to a broad range of robotic manipulation
tasks. This is partly due to the fact that reinforcement learning algorithms
are notoriously difficult and time consuming to train, which is exacerbated
when training from images rather than full-state inputs. As humans perform
manipulation tasks, our eyes closely monitor every step of the process with our
gaze focusing sequentially on the objects being manipulated. With this in mind,
we present our Attention-driven Robotic Manipulation (ARM) algorithm, which is
a general manipulation algorithm that can be applied to a range of
sparse-rewarded tasks, given only a small number of demonstrations. ARM splits
the complex task of manipulation into a 3 stage pipeline: (1) a Q-attention
agent extracts relevant pixel locations from RGB and point cloud inputs, (2) a
next-best pose agent that accepts crops from the Q-attention agent and outputs
poses, and (3) a control agent that takes the goal pose and outputs joint
actions. We show that current learning algorithms fail on a range of RLBench
tasks, whilst ARM is successful.

利用注意力驱动的机器人操作算法（ARM），结合 Q-attention 代理、姿态代理和控制代理构成的三阶段流水线，能够成功解决目前强化学习在处理机器人操作任务时存在的困难，特别是针对图像输入的情况，对 RLBench 标准任务有显著的提高。