For further progress in video object segmentation (VOS), larger, more
diverse, and more challenging datasets will be necessary. However, densely
labeling every frame with pixel masks does not scale to large datasets. We use
a deep convolutional network to automatically create pseudo-labels on a pixel
level from much cheaper bounding box annotations and investigate how far such
pseudo-labels can carry us for training state-of-the-art VOS approaches. A very
encouraging result of our study is that adding a manually annotated mask in
only a single video frame for each object is sufficient to generate
pseudo-labels which can be used to train a VOS method to reach almost the same
performance level as when training with fully segmented videos. We use this
workflow to create pixel pseudo-labels for the training set of the challenging
tracking dataset TAO, and we manually annotate a subset of the validation set.
Together, we obtain the new TAO-VOS benchmark, which we make publicly available
at www.vision.rwth-aachen.de/page/taovos. While the performance of
state-of-the-art methods on existing datasets starts to saturate, TAO-VOS
remains very challenging for current algorithms and reveals their shortcomings.

本研究通过使用深度卷积神经网络，基于较为便宜的边界框注释自动生成像素级别的伪标签；进而证明添加单个手工标注的掩模图像帧可以对训练视频目标分割（VOS）方法产生足够的影响，从而可以利用这些伪标签进行训练。我们将这种方法应用在 TAO 观察数据集，得到了一个挑战性的新的 TAO-VOS 基准数据集，为当前算法的不足揭示出明显的改进空间。