Referring video object segmentation (RVOS) aims to segment the target
instance referred by a given text expression in a video clip. The text
expression normally contains sophisticated descriptions of the instance's
appearance, actions, and relations with others. It is therefore rather
difficult for an RVOS model to capture all these attributes correspondingly in
the video; in fact, the model often favours more on the action- and
relation-related visual attribute of the instance. This can end up with
incomplete or even incorrect mask prediction of the target instance. In this
paper, we tackle this problem by taking a subject-centric short text expression
from the original long text expression. The short one retains only the
appearance-related information of the target instance so that we can use it to
focus the model's attention on the instance's appearance. We let the model make
joint predictions using both long and short text expressions and introduce a
long-short predictions intersection loss to align the joint predictions.
Besides the improvement on the linguistic part, we also introduce a
forward-backward visual consistency loss, which utilizes optical flows to warp
visual features between the annotated frames and their temporal neighbors for
consistency. We build our method on top of two state of the art
transformer-based pipelines for end-to-end training. Extensive experiments on
A2D-Sentences and JHMDB-Sentences datasets show impressive improvements of our
method.

本文提出了一种基于短文本表达的参考视频对象分割方法，结合长短文本表达进行联合预测，并引入了一种前向 - 后向视觉一致性损失，实现了在 A2D-Sentences 和 JHMDB-Sentences 数据集上表现出的显著性改进。