The objective of this paper is self-supervised learning of feature embeddings
that are suitable for matching correspondences along the videos, which we term
correspondence flow. By leveraging the natural spatial-temporal coherence in
videos, we propose to train a ``pointer'' that reconstructs a target frame by
copying pixels from a reference frame.
We make the following contributions: First, we introduce a simple information
bottleneck that forces the model to learn robust features for correspondence
matching, and prevent it from learning trivial solutions, \eg matching based on
low-level colour information. Second, to tackle the challenges from tracker
drifting, due to complex object deformations, illumination changes and
occlusions, we propose to train a recursive model over long temporal windows
with scheduled sampling and cycle consistency. Third, we achieve
state-of-the-art performance on DAVIS 2017 video segmentation and JHMDB
keypoint tracking tasks, outperforming all previous self-supervised learning
approaches by a significant margin. Fourth, in order to shed light on the
potential of self-supervised learning on the task of video correspondence flow,
we probe the upper bound by training on additional data, \ie more diverse
videos, further demonstrating significant improvements on video segmentation.

本文提出了一种适用于匹配视频中对应点的特征嵌入的自监督学习方法，其中使用自然的时空一致性训练指针模型，并通过引入信息瓶颈和循环模型等方式来解决跟踪器漂移等挑战，最终取得了在视频分割和关键点跟踪上的最佳效果。