Learning matching costs has been shown to be critical to the success of the
state-of-the-art deep stereo matching methods, in which 3D convolutions are
applied on a 4D feature volume to learn a 3D cost volume. However, this
mechanism has never been employed for the optical flow task. This is mainly due
to the significantly increased search dimension in the case of optical flow
computation, ie, a straightforward extension would require dense 4D
convolutions in order to process a 5D feature volume, which is computationally
prohibitive. This paper proposes a novel solution that is able to bypass the
requirement of building a 5D feature volume while still allowing the network to
learn suitable matching costs from data. Our key innovation is to decouple the
connection between 2D displacements and learn the matching costs at each 2D
displacement hypothesis independently, ie, displacement-invariant cost
learning. Specifically, we apply the same 2D convolution-based matching net
independently on each 2D displacement hypothesis to learn a 4D cost volume.
Moreover, we propose a displacement-aware projection layer to scale the learned
cost volume, which reconsiders the correlation between different displacement
candidates and mitigates the multi-modal problem in the learned cost volume.
The cost volume is then projected to optical flow estimation through a 2D
soft-argmin layer. Extensive experiments show that our approach achieves
state-of-the-art accuracy on various datasets, and outperforms all published
optical flow methods on the Sintel benchmark.

本研究基于深度立体匹配中使用的 3D 卷积学习 3D 成本体积和流形成本之间的映射关系，旨在提出一种新的算法，该算法绕过了需要构建 5D 特征体积的要求。具体而言，本研究提出解耦 2D 位移之间的关系，并在每个 2D 位移假设上独立地应用 2D 卷积匹配网络来学习 4D 成本体积，从而实现了独立于位移的成本学习。最后，我们采用 2D soft-argmin 层将成本体积投影到光流估计中。