Referring Video Object Segmentation (RVOS) aims to segment the object
referred to by the query sentence throughout the entire video. Most existing
methods require end-to-end training with dense mask annotations, which could be
computation-consuming and less scalable. In this work, we aim to efficiently
adapt foundation segmentation models for addressing RVOS from weak supervision
with the proposed Grounded Prompting (GroPrompt) framework. More specifically,
we propose Text-Aware Prompt Contrastive Learning (TAP-CL) to enhance the
association between the position prompts and the referring sentences with only
box supervisions, including Text-Contrastive Prompt Learning (TextCon) and
Modality-Contrastive Prompt Learning (ModalCon) at frame level and video level,
respectively. With the proposed TAP-CL, our GroPrompt framework can generate
temporal-consistent yet text-aware position prompts describing locations and
movements for the referred object from the video. The experimental results in
the standard RVOS benchmarks (Ref-YouTube-VOS, Ref-DAVIS17, A2D-Sentences, and
JHMDB-Sentences) demonstrate the competitive performance of our proposed
GroPrompt framework given only bounding box weak supervisions.

该研究提出了一种基于 Grounded Prompting (GroPrompt) 框架的高效适应基础分割模型来解决仅通过弱监督信息进行 Referring Video Object Segmentation (RVOS) 的问题，并提出了 Text-Aware Prompt Contrastive Learning (TAP-CL) 方法来增强位置提示和指代句子之间的关联。实验证明，该 GroPrompt 框架在标准 RVOS 基准测试中表现出竞争力，仅需边界框的弱监督信息即可。