Visual control policies can encounter significant performance degradation
when visual conditions like lighting or camera position differ from those seen
during training -- often exhibiting sharp declines in capability even for minor
differences. In this work, we examine robustness to a suite of these types of
visual changes for RGB-D and point cloud based visual control policies. To
perform these experiments on both model-free and model-based reinforcement
learners, we introduce a novel Point Cloud World Model (PCWM) and point cloud
based control policies. Our experiments show that policies that explicitly
encode point clouds are significantly more robust than their RGB-D
counterparts. Further, we find our proposed PCWM significantly outperforms
prior works in terms of sample efficiency during training. Taken together,
these results suggest reasoning about the 3D scene through point clouds can
improve performance, reduce learning time, and increase robustness for robotic
learners. Project Webpage: this https URL

通过对 RGB-D 和基于点云的视觉控制策略进行实验，我们发现明确编码点云的策略在可视条件变化下显著更加稳健，同时我们提出的点云世界模型（PCWM）在训练过程中大幅提升样本效率，这些结果表明通过点云对三维场景进行推理可以改善性能、缩短学习时间并提高机器学习算法的稳健性。

点云模型提升机器学习机器人的视觉鲁棒性

Point Cloud Models Improve Visual Robustness in Robotic Learners

In this report we review memory-based meta-learning as a tool for building
sample-efficient strategies that learn from past experience to adapt to any
task within a target class. Our goal is to equip the reader with the conceptual
foundations of this tool for building new, scalable agents that operate on
broad domains. To do so, we present basic algorithmic templates for building
near-optimal predictors and reinforcement learners which behave as if they had
a probabilistic model that allowed them to efficiently exploit task structure.
Furthermore, we recast memory-based meta-learning within a Bayesian framework,
showing that the meta-learned strategies are near-optimal because they amortize
Bayes-filtered data, where the adaptation is implemented in the memory dynamics
as a state-machine of sufficient statistics. Essentially, memory-based
meta-learning translates the hard problem of probabilistic sequential inference
into a regression problem.

在这篇论文中，我们回顾了基于记忆的元学习作为构建在目标类中适应任何任务的节约样本策略的工具，并通过基本算法模板提供构建近乎最优预测器和强化学习器的概念基础。此外，我们将记忆性元学习重新放入贝叶斯框架内，展示元学习策略接近最优，因为它们分摊了贝叶斯滤波之后的数据，其中适应在内存动态中实现为足够统计的状态机。基本上，基于记忆的元学习将概率序列推断的困难问题转化为回归问题。