Developing generalizable manipulation skills is a core challenge in embodied
AI. This includes generalization across diverse task configurations,
encompassing variations in object shape, density, friction coefficient, and
external disturbances such as forces applied to the robot. Rapid Motor
Adaptation (RMA) offers a promising solution to this challenge. It posits that
essential hidden variables influencing an agent's task performance, such as
object mass and shape, can be effectively inferred from the agent's action and
proprioceptive history. Drawing inspiration from RMA in locomotion and in-hand
rotation, we use depth perception to develop agents tailored for rapid motor
adaptation in a variety of manipulation tasks. We evaluated our agents on four
challenging tasks from the Maniskill2 benchmark, namely pick-and-place
operations with hundreds of objects from the YCB and EGAD datasets, peg
insertion with precise position and orientation, and operating a variety of
faucets and handles, with customized environment variations. Empirical results
demonstrate that our agents surpass state-of-the-art methods like automatic
domain randomization and vision-based policies, obtaining better generalization
performance and sample efficiency.

研究表明，利用深度感知开发在各种操作任务中迅速适应的代理人可以在实现更好的通用性能和样本效率方面超越自动领域随机化和基于视觉策略的最新方法。

机器人机械臂的快速运动适应

Rapid Motor Adaptation for Robotic Manipulator Arms

Generalizable manipulation skills, which can be composed to tackle
long-horizon and complex daily chores, are one of the cornerstones of Embodied
AI. However, existing benchmarks, mostly composed of a suite of simulatable
environments, are insufficient to push cutting-edge research works because they
lack object-level topological and geometric variations, are not based on fully
dynamic simulation, or are short of native support for multiple types of
manipulation tasks. To this end, we present ManiSkill2, the next generation of
the SAPIEN ManiSkill benchmark, to address critical pain points often
encountered by researchers when using benchmarks for generalizable manipulation
skills. ManiSkill2 includes 20 manipulation task families with 2000+ object
models and 4M+ demonstration frames, which cover stationary/mobile-base,
single/dual-arm, and rigid/soft-body manipulation tasks with 2D/3D-input data
simulated by fully dynamic engines. It defines a unified interface and
evaluation protocol to support a wide range of algorithms (e.g., classic
sense-plan-act, RL, IL), visual observations (point cloud, RGBD), and
controllers (e.g., action type and parameterization). Moreover, it empowers
fast visual input learning algorithms so that a CNN-based policy can collect
samples at about 2000 FPS with 1 GPU and 16 processes on a regular workstation.
It implements a render server infrastructure to allow sharing rendering
resources across all environments, thereby significantly reducing memory usage.
We open-source all codes of our benchmark (simulator, environments, and
baselines) and host an online challenge open to interdisciplinary researchers.

ManiSkill2 是 SAPIEN ManiSkill 基准的下一代版本，它包含 20 个操作任务族，2000 多个物体模型和 4M 多演示帧，定义了一个统一的界面和评估协议，支持各种算法，视觉观察和控制器，并赋予快速视觉输入学习算法的能力。