In this study, we explore the influence of different observation spaces on
robot learning, focusing on three predominant modalities: RGB, RGB-D, and point
cloud. Through extensive experimentation on over 17 varied contact-rich
manipulation tasks, conducted across two benchmarks and simulators, we have
observed a notable trend: point cloud-based methods, even those with the
simplest designs, frequently surpass their RGB and RGB-D counterparts in
performance. This remains consistent in both scenarios: training from scratch
and utilizing pretraining. Furthermore, our findings indicate that point cloud
observations lead to improved policy zero-shot generalization in relation to
various geometry and visual clues, including camera viewpoints, lighting
conditions, noise levels and background appearance. The outcomes suggest that
3D point cloud is a valuable observation modality for intricate robotic tasks.
We will open-source all our codes and checkpoints, hoping that our insights can
help design more generalizable and robust robotic models.

通过在 17 个不同的接触丰富的操作任务上进行广泛实验证明，针对机器人学习中的不同观察空间，特别是 RGB、RGB-D 和点云，点云方法在性能上通常超过 RGB 和 RGB-D，不论是从头开始训练还是利用预训练。此外，研究结果表明，点云观察对于各种几何和视觉线索（包括摄像机视角、光照条件、噪声水平和背景外观）的零样本泛化具有改进作用，暗示了 3D 点云作为一种有价值的观察模态在复杂的机器人任务中的应用。

点云的重要性：重新思考不同观测空间对机器人学习的影响

Point Cloud Matters: Rethinking the Impact of Different Observation  Spaces on Robot Learning

Simulating trajectories of virtual crowds is a commonly encountered task in
Computer Graphics. Several recent works have applied Reinforcement Learning
methods to animate virtual agents, however they often make different design
choices when it comes to the fundamental simulation setup. Each of these
choices comes with a reasonable justification for its use, so it is not obvious
what is their real impact, and how they affect the results. In this work, we
analyze some of these arbitrary choices in terms of their impact on the
learning performance, as well as the quality of the resulting simulation
measured in terms of the energy efficiency. We perform a theoretical analysis
of the properties of the reward function design, and empirically evaluate the
impact of using certain observation and action spaces on a variety of
scenarios, with the reward function and energy usage as metrics. We show that
directly using the neighboring agents' information as observation generally
outperforms the more widely used raycasting. Similarly, using nonholonomic
controls with egocentric observations tends to produce more efficient behaviors
than holonomic controls with absolute observations. Each of these choices has a
significant, and potentially nontrivial impact on the results, and so
researchers should be mindful about choosing and reporting them in their work.

本文旨在通过理论分析和实证实验，研究网络虚拟角色动画中强化学习方法在不同情景下表现的因素，并指出观察空间和奖励函数设计两个因素在网络虚拟角色动画中会产生显著的影响。