Previous works on video object segmentation (VOS) are trained on densely
annotated videos. Nevertheless, acquiring annotations in pixel level is
expensive and time-consuming. In this work, we demonstrate the feasibility of
training a satisfactory VOS model on sparsely annotated videos-we merely
require two labeled frames per training video while the performance is
sustained. We term this novel training paradigm as two-shot video object
segmentation, or two-shot VOS for short. The underlying idea is to generate
pseudo labels for unlabeled frames during training and to optimize the model on
the combination of labeled and pseudo-labeled data. Our approach is extremely
simple and can be applied to a majority of existing frameworks. We first
pre-train a VOS model on sparsely annotated videos in a semi-supervised manner,
with the first frame always being a labeled one. Then, we adopt the pre-trained
VOS model to generate pseudo labels for all unlabeled frames, which are
subsequently stored in a pseudo-label bank. Finally, we retrain a VOS model on
both labeled and pseudo-labeled data without any restrictions on the first
frame. For the first time, we present a general way to train VOS models on
two-shot VOS datasets. By using 7.3% and 2.9% labeled data of YouTube-VOS and
DAVIS benchmarks, our approach achieves comparable results in contrast to the
counterparts trained on fully labeled set. Code and models are available at
this https URL

本文提出了一种新的视频对象分割训练范式 - 两张标记帧的视频对象分割，通过在训练期间生成伪标签并将其与已标记数据结合优化模型，我们仅需要稀疏注释就可以训练出令人满意的模型。通过使用 YouTube-VOS 和 DAVIS 基准测试的仅仅 7.3％和 2.9％标记数据，我们的方法取得了与全标记集训练的同行相当的结果。