Pixel-level Video Understanding in the Wild Challenge (PVUW) focus on complex
video understanding. In this CVPR 2024 workshop, we add two new tracks, Complex
Video Object Segmentation Track based on MOSE dataset and Motion Expression
guided Video Segmentation track based on MeViS dataset. In the two new tracks,
we provide additional videos and annotations that feature challenging elements,
such as the disappearance and reappearance of objects, inconspicuous small
objects, heavy occlusions, and crowded environments in MOSE. Moreover, we
provide a new motion expression guided video segmentation dataset MeViS to
study the natural language-guided video understanding in complex environments.
These new videos, sentences, and annotations enable us to foster the
development of a more comprehensive and robust pixel-level understanding of
video scenes in complex environments and realistic scenarios. The MOSE
challenge had 140 registered teams in total, 65 teams participated the
validation phase and 12 teams made valid submissions in the final challenge
phase. The MeViS challenge had 225 registered teams in total, 50 teams
participated the validation phase and 5 teams made valid submissions in the
final challenge phase.

复杂环境下像素级视频理解的挑战，提供了基于 MOSE 数据集的复杂视频对象分割以及基于 MeViS 数据集的运动表达引导的视频分割两个新的跟踪，并通过提供具有挑战性元素的额外视频和注释来促进像素级视频场景的综合和强大的理解。

复杂视频理解的 PVUW 2024 挑战：方法与结果

PVUW 2024 Challenge on Complex Video Understanding: Methods and Results

We present Cutie, a video object segmentation (VOS) network with object-level
memory reading, which puts the object representation from memory back into the
video object segmentation result. Recent works on VOS employ bottom-up
pixel-level memory reading which struggles due to matching noise, especially in
the presence of distractors, resulting in lower performance in more challenging
data. In contrast, Cutie performs top-down object-level memory reading by
adapting a small set of object queries for restructuring and interacting with
the bottom-up pixel features iteratively with a query-based object transformer
(qt, hence Cutie). The object queries act as a high-level summary of the target
object, while high-resolution feature maps are retained for accurate
segmentation. Together with foreground-background masked attention, Cutie
cleanly separates the semantics of the foreground object from the background.
On the challenging MOSE dataset, Cutie improves by 8.7 J&F over XMem with a
similar running time and improves by 4.2 J&F over DeAOT while running three
times as fast. Code is available at: this https URL

Cutie 是一个视频目标分割网络，具有对象级记忆读取，通过适应一小组对象查询与自底向上的像素特征进行迭代交互，结合前景 - 背景屏蔽注意力，可准确分离前景对象的语义，从而在 MOSE 数据集上显著提高 J&F 值。