This paper presents a self-supervised method for learning reliable visual
correspondence from unlabeled videos. We formulate the correspondence as
finding paths in a joint space-time graph, where nodes are grid patches sampled
from frames, and are linked by two types of edges: (i) neighbor relations that
determine the aggregation strength from intra-frame neighbors in space, and
(ii) similarity relations that indicate the transition probability of
inter-frame paths across time. Leveraging the cycle-consistency in videos, our
contrastive learning objective discriminates dynamic objects from both their
neighboring views and temporal views. Compared with prior works, our approach
actively explores the neighbor relations of central instances to learn a latent
association between center-neighbor pairs (e.g., "hand -- arm") across time,
thus improving the instance discrimination. Without fine-tuning, our learned
representation outperforms the state-of-the-art self-supervised methods on a
variety of visual tasks including video object propagation, part propagation,
and pose keypoint tracking. Our self-supervised method also surpasses some
fully supervised algorithms designed for the specific tasks.

本研究提出了一种自监督学习方法，从未标记的视频中学习可靠的视觉对应关系。 所提出的方法将对应关系形式化为在联合空间 - 时间图中寻找路径的过程，并在此基础上利用循环一致性来识别图像中的动态对象，从而实现了中心 - 邻居对之间的潜在关联学习。实验结果表明，该方法在视频对象传播、部分传播和姿态关键点跟踪等各种视觉任务上均表现出色，并且优于某些针对特定任务设计的全监督算法。