Referring Video Object Segmentation (R-VOS) methods face challenges in
maintaining consistent object segmentation due to temporal context variability
and the presence of other visually similar objects. We propose an end-to-end
R-VOS paradigm that explicitly models temporal instance consistency alongside
the referring segmentation. Specifically, we introduce a novel hybrid memory
that facilitates inter-frame collaboration for robust spatio-temporal matching
and propagation. Features of frames with automatically generated high-quality
reference masks are propagated to segment the remaining frames based on
multi-granularity association to achieve temporally consistent R-VOS.
Furthermore, we propose a new Mask Consistency Score (MCS) metric to evaluate
the temporal consistency of video segmentation. Extensive experiments
demonstrate that our approach enhances temporal consistency by a significant
margin, leading to top-ranked performance on popular R-VOS benchmarks, i.e.,
Ref-YouTube-VOS (67.1%) and Ref-DAVIS17 (65.6%).

提出了一种新的端到端的 R-VOS 范例，通过显式建模时序实例一致性与指定分割相结合，引入了一种新颖的混合内存，用于在鲁棒的时空匹配和传播中促进帧间协作；通过多粒度的关联将具有高质量参考掩码的帧的特征传播到其余帧，以实现时间一致的 R-VOS；此外，提出了一种新的掩码一致性评分 (MCS) 指标，用于评估视频分割的时间一致性；大量实验证明我们的方法显著提高了时间一致性，在流行的 R-VOS 基准测试中取得了最高的性能，即 Ref-YouTube-VOS (67.1%) 和 Ref-DAVIS17 (65.6%)。

朝向时间一致的参考视频对象分割

Towards Temporally Consistent Referring Video Object Segmentation

We propose a self-supervised spatio-temporal matching method, coined
Motion-Aware Mask Propagation (MAMP), for video object segmentation. MAMP
leverages the frame reconstruction task for training without the need for
annotations. During inference, MAMP extracts high-resolution features from each
frame to build a memory bank from the features as well as the predicted masks
of selected past frames. MAMP then propagates the masks from the memory bank to
subsequent frames according to our proposed motion-aware spatio-temporal
matching module to handle fast motion and long-term matching scenarios.
Evaluation on DAVIS-2017 and YouTube-VOS datasets show that MAMP achieves
state-of-the-art performance with stronger generalization ability compared to
existing self-supervised methods, i.e., 4.2% higher mean J&F on DAVIS-2017 and
4.85% higher mean J&F on the unseen categories of YouTube-VOS than the nearest
competitor. Moreover, MAMP performs at par with many supervised video object
segmentation methods. Our code is available at:
this https URL

本文提出了一种基于自监督学习的时空匹配方法，称为 Motion-Aware Mask Propagation（MAMP），用于视频对象分割，利用帧重建任务进行训练，而无需进行标注，在 DAVIS-2017 和 YouTube-VOS 数据集上得到了最先进的性能。