Many complex robotic manipulation tasks can be decomposed as a sequence of
pick and place actions. Training a robotic agent to learn this sequence over
many different starting conditions typically requires many iterations or
demonstrations, especially in 3D environments. In this work, we propose Fourier
Transporter (\ours{}) which leverages the two-fold $\SE(d)\times\SE(d)$
symmetry in the pick-place problem to achieve much higher sample efficiency.
\ours{} is an open-loop behavior cloning method trained using expert
demonstrations to predict pick-place actions on new environments. \ours{} is
constrained to incorporate symmetries of the pick and place actions
independently. Our method utilizes a fiber space Fourier transformation that
allows for memory-efficient construction. We test our proposed network on the
RLbench benchmark and achieve state-of-the-art results across various tasks.

利用傅里叶转换器在复杂机器人操作任务中提高样本效率，通过训练和预测新环境下的拾取 - 放置动作来实现开环行为克隆方法，并独立考虑拾取和放置动作的对称性，取得了在 RLbench 基准测试中的最新成果。

傅立叶传输器：三维双等变机器人操作

Fourier Transporter: Bi-Equivariant Robotic Manipulation in 3D

The ability for robots to comprehend and execute manipulation tasks based on
natural language instructions is a long-term goal in robotics. The dominant
approaches for language-guided manipulation use 2D image representations, which
face difficulties in combining multi-view cameras and inferring precise 3D
positions and relationships. To address these limitations, we propose a 3D
point cloud based policy called PolarNet for language-guided manipulation. It
leverages carefully designed point cloud inputs, efficient point cloud
encoders, and multimodal transformers to learn 3D point cloud representations
and integrate them with language instructions for action prediction. PolarNet
is shown to be effective and data efficient in a variety of experiments
conducted on the RLBench benchmark. It outperforms state-of-the-art 2D and 3D
approaches in both single-task and multi-task learning. It also achieves
promising results on a real robot.

通过使用 3D 点云来进行语言引导的操纵任务，我们提出了一种名为 PolarNet 的策略，该策略通过精心设计的点云输入、高效的点云编码器和多模态变换器来学习 3D 点云表示并将其与语言指令集成以进行行动预测。在 RLBench 基准测试中，PolarNet 在各种实验中显示出了高效和数据高效性，并在单任务和多任务学习中优于最先进的 2D 和 3D 方法。它在实际机器人上也取得了令人期待的结果。

PolarNet: 基于语言引导的机器人操作的 3D 点云

PolarNet: 3D Point Clouds for Language-Guided Robotic Manipulation

In human environments, robots are expected to accomplish a variety of
manipulation tasks given simple natural language instructions. Yet, robotic
manipulation is extremely challenging as it requires fine-grained motor
control, long-term memory as well as generalization to previously unseen tasks
and environments. To address these challenges, we propose a unified
transformer-based approach that takes into account multiple inputs. In
particular, our transformer architecture integrates (i) natural language
instructions and (ii) multi-view scene observations while (iii) keeping track
of the full history of observations and actions. Such an approach enables
learning dependencies between history and instructions and improves
manipulation precision using multiple views. We evaluate our method on the
challenging RLBench benchmark and on a real-world robot. Notably, our approach
scales to 74 diverse RLBench tasks and outperforms the state of the art. We
also address instruction-conditioned tasks and demonstrate excellent
generalization to previously unseen variations.

本研究提出了一种基于 Transformer 的方法来处理人类语言指令和多视角场景观察，以提高机器人的精准操作，并在 RLBench 基准测试中成功实现 74 项任务，还表现出对未曾见过变化的任务的良好泛化性能。