Pixel-level Video Understanding in the Wild Challenge (PVUW) focus on complex
video understanding. In this CVPR 2024 workshop, we add two new tracks, Complex
Video Object Segmentation Track based on MOSE dataset and Motion Expression
guided Video Segmentation track based on MeViS dataset. In the two new tracks,
we provide additional videos and annotations that feature challenging elements,
such as the disappearance and reappearance of objects, inconspicuous small
objects, heavy occlusions, and crowded environments in MOSE. Moreover, we
provide a new motion expression guided video segmentation dataset MeViS to
study the natural language-guided video understanding in complex environments.
These new videos, sentences, and annotations enable us to foster the
development of a more comprehensive and robust pixel-level understanding of
video scenes in complex environments and realistic scenarios. The MOSE
challenge had 140 registered teams in total, 65 teams participated the
validation phase and 12 teams made valid submissions in the final challenge
phase. The MeViS challenge had 225 registered teams in total, 50 teams
participated the validation phase and 5 teams made valid submissions in the
final challenge phase.

复杂环境下像素级视频理解的挑战，提供了基于 MOSE 数据集的复杂视频对象分割以及基于 MeViS 数据集的运动表达引导的视频分割两个新的跟踪，并通过提供具有挑战性元素的额外视频和注释来促进像素级视频场景的综合和强大的理解。

复杂视频理解的 PVUW 2024 挑战：方法与结果

PVUW 2024 Challenge on Complex Video Understanding: Methods and Results

This paper strives for motion expressions guided video segmentation, which
focuses on segmenting objects in video content based on a sentence describing
the motion of the objects. Existing referring video object datasets typically
focus on salient objects and use language expressions that contain excessive
static attributes that could potentially enable the target object to be
identified in a single frame. These datasets downplay the importance of motion
in video content for language-guided video object segmentation. To investigate
the feasibility of using motion expressions to ground and segment objects in
videos, we propose a large-scale dataset called MeViS, which contains numerous
motion expressions to indicate target objects in complex environments. We
benchmarked 5 existing referring video object segmentation (RVOS) methods and
conducted a comprehensive comparison on the MeViS dataset. The results show
that current RVOS methods cannot effectively address motion expression-guided
video segmentation. We further analyze the challenges and propose a baseline
approach for the proposed MeViS dataset. The goal of our benchmark is to
provide a platform that enables the development of effective language-guided
video segmentation algorithms that leverage motion expressions as a primary cue
for object segmentation in complex video scenes. The proposed MeViS dataset has
been released at this https URL

本研究旨在使用运动表达引导的视频分割，通过在描述对象运动的句子上对视频内容中的对象进行分割。我们提出了一个名为 MeViS 的大规模数据集，其中包含了大量的运动表达，用于指示复杂环境中的目标对象。通过对 MeViS 数据集进行 5 种现有的参考视频对象分割方法进行基准测试和全面比较，结果表明目前的方法不能有效地处理运动表达引导的视频分割。我们进一步分析了所面临的挑战，并在 MeViS 数据集上提出了一个基准方法。我们的目标是提供一个平台，以便开发对复杂视频场景中的对象分割利用运动表达作为主要线索的有效语言引导视频分割算法。MeViS 数据集已在此 URL 发布。