Previous cycle-consistency correspondence learning methods usually leverage
image patches for training. In this paper, we present a fully convolutional
method, which is simpler and more coherent to the inference process. While
directly applying fully convolutional training results in model collapse, we
study the underline reason behind this collapse phenomenon, indicating that the
absolute positions of pixels provide a shortcut to easily accomplish
cycle-consistence, which hinders the learning of meaningful visual
representations. To break this absolute position shortcut, we propose to apply
different crops for forward and backward frames, and adopt feature warping to
establish correspondence between two crops of a same frame. The former
technique enforces the corresponding pixels at forward and back tracks to have
different absolute positions, and the latter effectively blocks the shortcuts
going between forward and back tracks. In three label propagation benchmarks
for pose tracking, face landmark tracking and video object segmentation, our
method largely improves the results of vanilla fully convolutional
cycle-consistency method, achieving very competitive performance compared with
the self-supervised state-of-the-art approaches. Our trained model and code are
available at https://github.com/Steve-Tod/STFC3.

本文提出了一种全卷积的方法，采用前向和后向帧之间的不同裁剪策略、特征变形等技术，破解绝对位置快捷方式，有效地阻止了前向和后向跟踪之间的快捷方式，并在三个标签传播基准测试中实现很好的性能。

探究基于全卷积循环一致性的视频对应关系学习

Breaking Shortcut: Exploring Fully Convolutional Cycle-Consistency for  Video Correspondence Learning

Feature warping is a core technique in optical flow estimation; however, the
ambiguity caused by occluded areas during warping is a major problem that
remains unsolved. In this paper, we propose an asymmetric occlusion-aware
feature matching module, which can learn a rough occlusion mask that filters
useless (occluded) areas immediately after feature warping without any explicit
supervision. The proposed module can be easily integrated into end-to-end
network architectures and enjoys performance gains while introducing negligible
computational cost. The learned occlusion mask can be further fed into a
subsequent network cascade with dual feature pyramids with which we achieve
state-of-the-art performance. At the time of submission, our method, called
MaskFlownet, surpasses all published optical flow methods on the MPI Sintel,
KITTI 2012 and 2015 benchmarks. Code is available at
this https URL

本文提出了一种不对 occlusion 区域进行显式监督学习的不对称 occlusion-aware 特征匹配模块。此模块可以在特征变形之后学习粗略的 occlusion 掩蔽，可以轻松地与端到端网络架构集成，并在不引入可忽略的计算成本的同时获得性能提升。学到的 occlusion 掩蔽可以进一步输入到具有双特征金字塔的后续网络级联中，其中我们实现了最先进的性能。我们的方法 MaskFlownet 在 MPI Sintel、KITTI 2012 和 2015 基准测试中均超过了所有已发布的光流方法。