Current state-of-the-art Video Object Segmentation (VOS) methods rely on
dense per-object mask annotations both during training and testing. This
requires time-consuming and costly video annotation mechanisms. We propose a
novel Point-VOS task with a spatio-temporally sparse point-wise annotation
scheme that substantially reduces the annotation effort. We apply our
annotation scheme to two large-scale video datasets with text descriptions and
annotate over 19M points across 133K objects in 32K videos. Based on our
annotations, we propose a new Point-VOS benchmark, and a corresponding
point-based training mechanism, which we use to establish strong baseline
results. We show that existing VOS methods can easily be adapted to leverage
our point annotations during training, and can achieve results close to the
fully-supervised performance when trained on pseudo-masks generated from these
points. In addition, we show that our data can be used to improve models that
connect vision and language, by evaluating it on the Video Narrative Grounding
(VNG) task. We will make our code and annotations available at
this https URL

提出了一种基于稀疏点注释方案的新型 Point-VOS 任务和标注机制，通过基于伪掩码的训练可以使现有的 VOS 方法在性能上接近全监督的表现，并且该数据可用于改进视觉和语言相关的模型。

点指上行视频目标分割

Point-VOS: Pointing Up Video Object Segmentation

We propose Video Localized Narratives, a new form of multimodal video
annotations connecting vision and language. In the original Localized
Narratives, annotators speak and move their mouse simultaneously on an image,
thus grounding each word with a mouse trace segment. However, this is
challenging on a video. Our new protocol empowers annotators to tell the story
of a video with Localized Narratives, capturing even complex events involving
multiple actors interacting with each other and with several passive objects.
We annotated 20k videos of the OVIS, UVO, and Oops datasets, totalling 1.7M
words. Based on this data, we also construct new benchmarks for the video
narrative grounding and video question answering tasks, and provide reference
results from strong baseline models. Our annotations are available at
this https URL

本文提出了一种新形式的多模态视频注释方法：基于视频的定位叙事。在此方法中，注释员不仅可以说出视频中发生的事件，而且还可以使用鼠标追踪分段来定位句子的表示部分。同时，作者采用了该方法对三个数据集中的 20,000 个视频进行了注释，并提供了视频故事理解和视频问题回答任务的评测基准以及来自强基准模型的参考结果。