Unsupervised multi-object segmentation has shown impressive results on images
by utilizing powerful semantics learned from self-supervised pretraining. An
additional modality such as depth or motion is often used to facilitate the
segmentation in video sequences. However, the performance improvements observed
in synthetic sequences, which rely on the robustness of an additional cue, do
not translate to more challenging real-world scenarios. In this paper, we
propose the first fully unsupervised method for segmenting multiple objects in
real-world sequences. Our object-centric learning framework spatially binds
objects to slots on each frame and then relates these slots across frames. From
these temporally-aware slots, the training objective is to reconstruct the
middle frame in a high-level semantic feature space. We propose a masking
strategy by dropping a significant portion of tokens in the feature space for
efficiency and regularization. Additionally, we address over-clustering by
merging slots based on similarity. Our method can successfully segment multiple
instances of complex and high-variety classes in YouTube videos.

本文提出了第一个完全无监督的方法，用于在真实场景的序列中分割多个对象，通过空间绑定对象并将这些槽联系起来，在高级语义特征空间中重建中间帧，解决了效率和规范化问题，成功地在 YouTube 视频中分割了复杂且多变的类别的多个实例。

视频的自监督目标中心学习

Self-supervised Object-Centric Learning for Videos

Recent work in unsupervised multi-object segmentation shows impressive
results by predicting motion from a single image despite the inherent ambiguity
in predicting motion without the next image. On the other hand, the set of
possible motions for an image can be constrained to a low-dimensional space by
considering the scene structure and moving objects in it. We propose to model
pixel-wise geometry and object motion to remove ambiguity in reconstructing
flow from a single image. Specifically, we divide the image into coherently
moving regions and use depth to construct flow bases that best explain the
observed flow in each region. We achieve state-of-the-art results in
unsupervised multi-object segmentation on synthetic and real-world datasets by
modeling the scene structure and object motion. Our evaluation of the predicted
depth maps shows reliable performance in monocular depth estimation.

本文提出一种基于像素几何和物体运动模型的运动预测方法，通过将图像分为运动一致的区域并利用深度构建最佳匹配的流场基础来减少从单个图像重建流场的不确定性，并在场景结构和物体运动建模方面取得了最新的结果，同时对预测深度图的评估显示出可靠的单眼深度估计性能。

低维物体运动多目标发现

Multi-Object Discovery by Low-Dimensional Object Motion

There has been a recent surge in methods that aim to decompose and segment
scenes into multiple objects in an unsupervised manner, i.e., unsupervised
multi-object segmentation. Performing such a task is a long-standing goal of
computer vision, offering to unlock object-level reasoning without requiring
dense annotations to train segmentation models. Despite significant progress,
current models are developed and trained on visually simple scenes depicting
mono-colored objects on plain backgrounds. The natural world, however, is
visually complex with confounding aspects such as diverse textures and
complicated lighting effects. In this study, we present a new benchmark called
ClevrTex, designed as the next challenge to compare, evaluate and analyze
algorithms. ClevrTex features synthetic scenes with diverse shapes, textures
and photo-mapped materials, created using physically based rendering
techniques. It includes 50k examples depicting 3-10 objects arranged on a
background, created using a catalog of 60 materials, and a further test set
featuring 10k images created using 25 different materials. We benchmark a large
set of recent unsupervised multi-object segmentation models on ClevrTex and
find all state-of-the-art approaches fail to learn good representations in the
textured setting, despite impressive performance on simpler data. We also
create variants of the ClevrTex dataset, controlling for different aspects of
scene complexity, and probe current approaches for individual shortcomings.
Dataset and code are available at
this https URL

该研究提出了一个名为 ClevrTex 的新基准，由具有多种形状和纹理的物体组成，并使用基于物理的渲染技术创建。作者将当前的无监督多对象分割模型应用于 ClevrTex 数据集，并发现现有的方法在纹理复杂的场景中无法学习良好的表示，即使对于简单数据的表现也非常出色。