Occlusion problem remains a key challenge in Optical Flow Estimation (OFE)
despite the recent significant progress brought by deep learning in the field.
Most existing deep learning OFE methods, especially those based on two frames,
cannot properly handle occlusions, in part because there is no significant
feature similarity in occluded regions. The multi-frame settings have the
potential to mitigate the occlusion issue in OFE. However, the problem of
Multi-frame OFE (MOFE) remains underexplored, and the limited works are
specially designed for pyramid backbones and obtain the aligned temporal
information by time-consuming backward flow calculation or non-differentiable
forward warping transformation. To address these shortcomings, we propose an
efficient MOFE framework named SplatFlow, which is realized by introducing the
differentiable splatting transformation to align the temporal information,
designing a One-to-Many embedding method to densely guide the current frame's
estimation, and further remodelling the existing two-frame backbones. The
proposed SplatFlow is very efficient yet more accurate as it is able to handle
occlusions properly. Extensive experimental evaluations show that our SplatFlow
substantially outperforms all published methods on KITTI2015 and Sintel
benchmarks. Especially on Sintel benchmark, SplatFlow achieves errors of 1.12
(clean pass) and 2.07 (final pass), with surprisingly significant 19.4% and
16.2% error reductions from the previous best results submitted, respectively.
Code is available at this https URL

本文中，我们提出了一种称为 SplatFlow 的 multi-frame optical flow 估计（mOFE）框架，可以通过引入可微变换方法来处理遮挡问题，并在 KITTI2015 和 Sintel 基准评估中得到显著的改进。