Self-supervised methods have shown remarkable progress in learning high-level
semantics and low-level temporal correspondence. Building on these results, we
take one step further and explore the possibility of integrating these two
features to enhance object-centric representations. Our preliminary experiments
indicate that query slot attention can extract different semantic components
from the RGB feature map, while random sampling based slot attention can
exploit temporal correspondence cues between frames to assist instance
identification. Motivated by this, we propose a novel semantic-aware masked
slot attention on top of the fused semantic features and correspondence maps.
It comprises two slot attention stages with a set of shared learnable Gaussian
distributions. In the first stage, we use the mean vectors as slot
initialization to decompose potential semantics and generate semantic
segmentation masks through iterative attention. In the second stage, for each
semantics, we randomly sample slots from the corresponding Gaussian
distribution and perform masked feature aggregation within the semantic area to
exploit temporal correspondence patterns for instance identification. We adopt
semantic- and instance-level temporal consistency as self-supervision to
encourage temporally coherent object-centric representations. Our model
effectively identifies multiple object instances with semantic structure,
reaching promising results on unsupervised video object discovery. Furthermore,
we achieve state-of-the-art performance on dense label propagation tasks,
demonstrating the potential for object-centric analysis. The code is released
at this https URL

自我监督方法在学习高层语义和低层时间对应方面取得了显著进展，本文在此基础上进一步探索了整合这两个特征以增强以对象为中心的表示的可能性。我们提出了一种新颖的语义感知遮蔽插槽注意力模型，通过融合语义特征和对应关系图，有效地识别多个对象实例，达到了在无监督视频对象发现和密集标签传播任务上的有希望结果，展示了以对象为中心的分析的潜力。