Interactive video object segmentation is a crucial video task, having various
applications from video editing to data annotating. However, current approaches
struggle to accurately segment objects across diverse domains. Recently,
Segment Anything Model (SAM) introduces interactive visual prompts and
demonstrates impressive performance across different domains. In this paper, we
propose a training-free prompt tracking framework for interactive video object
segmentation (I-PT), leveraging the powerful generalization of SAM. Although
point tracking efficiently captures the pixel-wise information of objects in a
video, points tend to be unstable when tracked over a long period, resulting in
incorrect segmentation. Towards fast and robust interaction, we jointly adopt
sparse points and boxes tracking, filtering out unstable points and capturing
object-wise information. To better integrate reference information from
multiple interactions, we introduce a cross-round space-time module (CRSTM),
which adaptively aggregates mask features from previous rounds and frames,
enhancing the segmentation stability. Our framework has demonstrated robust
zero-shot video segmentation results on popular VOS datasets with interaction
types, including DAVIS 2017, YouTube-VOS 2018, and MOSE 2023, maintaining a
good tradeoff between performance and interaction time.

我们提出了一种无需训练的提示跟踪框架，将稀疏点和框跟踪相结合，采用交叉循环时空模块从多次交互中自适应地聚合引用信息，实现了在流行的 VOS 数据集上稳定的零样本视频分割结果，保持了性能和交互时间之间的良好权衡。