In this paper, we present a unified, end-to-end trainable spatiotemporal CNN
model for VOS, which consists of two branches, i.e., the temporal coherence
branch and the spatial segmentation branch. Specifically, the temporal
coherence branch pretrained in an adversarial fashion from unlabeled video
data, is designed to capture the dynamic appearance and motion cues of video
sequences to guide object segmentation. The spatial segmentation branch focuses
on segmenting objects accurately based on the learned appearance and motion
cues. To obtain accurate segmentation results, we design a coarse-to-fine
process to sequentially apply a designed attention module on multi-scale
feature maps, and concatenate them to produce the final prediction. In this
way, the spatial segmentation branch is enforced to gradually concentrate on
object regions. These two branches are jointly fine-tuned on video segmentation
sequences in an end-to-end manner. Several experiments are carried out on three
challenging datasets (i.e., DAVIS-2016, DAVIS-2017 and Youtube-Object) to show
that our method achieves favorable performance against the state-of-the-arts.
Code is available at this https URL

本文提出了一种统一的、端到端可训练的时空卷积神经网络模型，用于视觉目标分割，包括时间一致分支和空间分割分支，并以优化的方式使用未标记的视频数据从中预训练时空一致性分支，以捕获视频序列的动态外观和运动线索引导对象分割。