Optical flow estimation is very challenging in situations with transparent or
occluded objects. In this work, we address these challenges at the task level
by introducing Amodal Optical Flow, which integrates optical flow with amodal
perception. Instead of only representing the visible regions, we define amodal
optical flow as a multi-layered pixel-level motion field that encompasses both
visible and occluded regions of the scene. To facilitate research on this new
task, we extend the AmodalSynthDrive dataset to include pixel-level labels for
amodal optical flow estimation. We present several strong baselines, along with
the Amodal Flow Quality metric to quantify the performance in an interpretable
manner. Furthermore, we propose the novel AmodalFlowNet as an initial step
toward addressing this task. AmodalFlowNet consists of a transformer-based
cost-volume encoder paired with a recurrent transformer decoder which
facilitates recurrent hierarchical feature propagation and amodal semantic
grounding. We demonstrate the tractability of amodal optical flow in extensive
experiments and show its utility for downstream tasks such as panoptic
tracking. We make the dataset, code, and trained models publicly available at
this http URL

通过引入 Amodal Optical Flow 来解决透明或被遮挡物体造成的光流估计挑战，将可见区域与被遮挡区域结合起来形成的多层级像素级运动场景，提出了 AmodalSynthDrive 数据集来帮助研究该任务，并提供了强基准、Amodal Flow Quality 度量和 AmodalFlowNet 模型作为解决该任务的初步尝试，并证明其在广泛实验中的易处理性和在诸如全景跟踪等下游任务中的实用性。