Video Object Segmentation (VOS) is typically formulated in a semi-supervised
setting. Given the ground-truth segmentation mask on the first frame, the task
of VOS is to track and segment the single or multiple objects of interests in
the rest frames of the video at the pixel level. One of the fundamental
challenges in VOS is how to make the most use of the temporal information to
boost the performance. We present an end-to-end network which stores short- and
long-term video sequence information preceding the current frame as the
temporal memories to address the temporal modeling in VOS. Our network consists
of two temporal sub-networks including a short-term memory sub-network and a
long-term memory sub-network. The short-term memory sub-network models the
fine-grained spatial-temporal interactions between local regions across
neighboring frames in video via a graph-based learning framework, which can
well preserve the visual consistency of local regions over time. The long-term
memory sub-network models the long-range evolution of object via a
Simplified-Gated Recurrent Unit (S-GRU), making the segmentation be robust
against occlusions and drift errors. In our experiments, we show that our
proposed method achieves a favorable and competitive performance on three
frequently-used VOS datasets, including DAVIS 2016, DAVIS 2017 and Youtube-VOS
in terms of both speed and accuracy.

本文提出了一种端到端的网络来解决视频对象分割中时间建模的根本挑战，包括短期记忆子网络和长期记忆子网络，通过图形学习框架提高局部区域间的可视一致性，在三个常用的数据集上取得了非常好的性能。