In human environments, robots are expected to accomplish a variety of
manipulation tasks given simple natural language instructions. Yet, robotic
manipulation is extremely challenging as it requires fine-grained motor
control, long-term memory as well as generalization to previously unseen tasks
and environments. To address these challenges, we propose a unified
transformer-based approach that takes into account multiple inputs. In
particular, our transformer architecture integrates (i) natural language
instructions and (ii) multi-view scene observations while (iii) keeping track
of the full history of observations and actions. Such an approach enables
learning dependencies between history and instructions and improves
manipulation precision using multiple views. We evaluate our method on the
challenging RLBench benchmark and on a real-world robot. Notably, our approach
scales to 74 diverse RLBench tasks and outperforms the state of the art. We
also address instruction-conditioned tasks and demonstrate excellent
generalization to previously unseen variations.

本研究提出了一种基于 Transformer 的方法来处理人类语言指令和多视角场景观察，以提高机器人的精准操作，并在 RLBench 基准测试中成功实现 74 项任务，还表现出对未曾见过变化的任务的良好泛化性能。