This paper strives for motion expressions guided video segmentation, which
focuses on segmenting objects in video content based on a sentence describing
the motion of the objects. Existing referring video object datasets typically
focus on salient objects and use language expressions that contain excessive
static attributes that could potentially enable the target object to be
identified in a single frame. These datasets downplay the importance of motion
in video content for language-guided video object segmentation. To investigate
the feasibility of using motion expressions to ground and segment objects in
videos, we propose a large-scale dataset called MeViS, which contains numerous
motion expressions to indicate target objects in complex environments. We
benchmarked 5 existing referring video object segmentation (RVOS) methods and
conducted a comprehensive comparison on the MeViS dataset. The results show
that current RVOS methods cannot effectively address motion expression-guided
video segmentation. We further analyze the challenges and propose a baseline
approach for the proposed MeViS dataset. The goal of our benchmark is to
provide a platform that enables the development of effective language-guided
video segmentation algorithms that leverage motion expressions as a primary cue
for object segmentation in complex video scenes. The proposed MeViS dataset has
been released at this https URL

本研究旨在使用运动表达引导的视频分割，通过在描述对象运动的句子上对视频内容中的对象进行分割。我们提出了一个名为 MeViS 的大规模数据集，其中包含了大量的运动表达，用于指示复杂环境中的目标对象。通过对 MeViS 数据集进行 5 种现有的参考视频对象分割方法进行基准测试和全面比较，结果表明目前的方法不能有效地处理运动表达引导的视频分割。我们进一步分析了所面临的挑战，并在 MeViS 数据集上提出了一个基准方法。我们的目标是提供一个平台，以便开发对复杂视频场景中的对象分割利用运动表达作为主要线索的有效语言引导视频分割算法。MeViS 数据集已在此 URL 发布。