This paper proposes to learn reliable dense correspondence from videos in a self-supervised manner. Our learning process integrates two highly related tasks: tracking large image regions \emph{and} establishing fine-grained pixel-level associations between consecutive video frames. We exploit the synergy between both tasks through a shared inter-frame affinity matrix, which simultaneously models transitions between video frames at both the region- and pixel-levels. While region-level localization helps reduce ambiguities in fine-grained matching by narrowing down search regions; fine-grained matching provides bottom-up features to facilitate region-level localization. Our method outperforms the state-of-the-art self-supervised methods on a variety of visual correspondence tasks, including video-object and part-segmentation propagation, keypoint tracking, and object tracking. Our self-supervised method even surpasses the fully-supervised affinity feature representation obtained from a ResNet-18 pre-trained on the ImageNet.

本文提出了一种利用自我监督方式从视频中学习可靠密集对应关系的方法，通过跟踪大规模图像区域和建立连续视频帧之间的像素级细粒度关联来实现。该方法利用共享的帧内亲和矩阵来建模两个任务之间的协同作用，在区域级别和像素级别同时建模视频帧之间的转换，从而在视觉对应任务中实现了优异的表现。

联合任务自监督学习用于时间对应