For further progress in video object segmentation (VOS), larger, more diverse, and more challenging datasets will be necessary. However, densely labeling every frame with pixel masks does not scale to large datasets. We use a deep convolutional network to automatically create pseudo-labels on a pixel level from much cheaper bounding box annotations and investigate how far such pseudo-labels can carry us for training state-of-the-art VOS approaches. A very encouraging result of our study is that adding a manually annotated mask in only a single video frame for each object is sufficient to generate pseudo-labels which can be used to train a VOS method to reach almost the same performance level as when training with fully segmented videos. We use this workflow to create pixel pseudo-labels for the training set of the challenging tracking dataset TAO, and we manually annotate a subset of the validation set. Together, we obtain the new TAO-VOS benchmark, which we make publicly available at www.vision.rwth-aachen.de/page/taovos. While the performance of state-of-the-art methods on existing datasets starts to saturate, TAO-VOS remains very challenging for current algorithms and reveals their shortcomings.

本研究通过使用深度卷积神经网络，基于较为便宜的边界框注释自动生成像素级别的伪标签；进而证明添加单个手工标注的掩模图像帧可以对训练视频目标分割（VOS）方法产生足够的影响，从而可以利用这些伪标签进行训练。我们将这种方法应用在 TAO 观察数据集，得到了一个挑战性的新的 TAO-VOS 基准数据集，为当前算法的不足揭示出明显的改进空间。

减少视频物体分割数据集注释工作量