There have recently been significant advances in the problem of unsupervised
object-centric representation learning and its application to downstream tasks.
The latest works support the argument that employing disentangled object
representations in image-based object-centric reinforcement learning tasks
facilitates policy learning. We propose a novel object-centric reinforcement
learning algorithm combining actor-critic and model-based approaches to utilize
these representations effectively. In our approach, we use a transformer
encoder to extract object representations and graph neural networks to
approximate the dynamics of an environment. The proposed method fills a
research gap in developing efficient object-centric world models for
reinforcement learning settings that can be used for environments with discrete
or continuous action spaces. Our algorithm performs better in a visually
complex 3D robotic environment and a 2D environment with compositional
structure than the state-of-the-art model-free actor-critic algorithm built
upon transformer architecture and the state-of-the-art monolithic model-based
algorithm.

最近在无监督的物体中心表示学习问题和其在下游任务中的应用方面取得了重大进展。最新研究支持这样一个论点：在基于图像的物体中心强化学习任务中使用解耦的物体表示有助于策略学习。我们提出了一种新颖的物体中心强化学习算法，结合了演员 - 评论家和基于模型的方法，有效地利用这些表示。我们的方法使用转换编码器提取物体表示，并使用图神经网络来近似环境动力学。所提出的方法填补了开发用于离散或连续动作空间环境的高效物体中心世界模型的研究空白。与基于转换器架构的最先进的无模型演员 - 评论家算法和最先进的整合模型为基础的算法相比，我们的算法在视觉复杂的三维机器人环境和具有组合结构的二维环境中表现更好。

图形对象中心的演员 - 评论家算法

Graphical Object-Centric Actor-Critic

We tackle the task of scalable unsupervised object-centric representation
learning on 3D scenes. Existing approaches to object-centric representation
learning show limitations in generalizing to larger scenes as their learning
processes rely on a fixed global coordinate system. In contrast, we propose to
learn view-invariant 3D object representations in localized object coordinate
systems. To this end, we estimate the object pose and appearance representation
separately and explicitly map object representations across views while
maintaining object identities. We adopt an amortized variational inference
pipeline that can process sequential input and scalably update object latent
distributions online. To handle large-scale scenes with a varying number of
objects, we further introduce a Cognitive Map that allows the registration and
query of objects on a per-scene global map to achieve scalable representation
learning. We explore the object-centric neural radiance field (NeRF) as our 3D
scene representation, which is jointly modeled within our unsupervised
object-centric learning framework. Experimental results on synthetic and real
datasets show that our proposed method can infer and maintain object-centric
representations of 3D scenes and outperforms previous models.

我们提出了一种可伸缩的无监督面向对象中心的 3D 场景表示学习方法，通过学习在局部对象坐标系统中视角不变的 3D 对象表示，从而推断和维护 3D 场景的对象中心表示，并优于以前的模型。

可扩展的三维对象为中心学习的变异推断

Variational Inference for Scalable 3D Object-centric Learning

Unsupervised object-centric representation (OCR) learning has recently drawn
attention as a new paradigm of visual representation. This is because of its
potential of being an effective pre-training technique for various downstream
tasks in terms of sample efficiency, systematic generalization, and reasoning.
Although image-based reinforcement learning (RL) is one of the most important
and thus frequently mentioned such downstream tasks, the benefit in RL has
surprisingly not been investigated systematically thus far. Instead, most of
the evaluations have focused on rather indirect metrics such as segmentation
quality and object property prediction accuracy. In this paper, we investigate
the effectiveness of OCR pre-training for image-based reinforcement learning
via empirical experiments. For systematic evaluation, we introduce a simple
object-centric visual RL benchmark and conduct experiments to answer questions
such as ``Does OCR pre-training improve performance on object-centric tasks?''
and ``Can OCR pre-training help with out-of-distribution generalization?''. Our
results provide empirical evidence for valuable insights into the effectiveness
of OCR pre-training for RL and the potential limitations of its use in certain
scenarios. Additionally, this study also examines the critical aspects of
incorporating OCR pre-training in RL, including performance in a visually
complex environment and the appropriate pooling layer to aggregate the object
representations.

本文通过实验探究并系统评估 OCR 预训练对于基于图像的强化学习原型任务的影响及其局限性，研究结果为在某些情况下 OCR 预训练使用的潜在局限性提供了实证证据，并分析了影响 OCR 预训练在强化学习中应用的关键因素，包括在视觉复杂环境中的性能和适当的汇聚层以聚合对象表示。