In existing Video Frame Interpolation (VFI) approaches, the motion estimation
between neighboring frames plays a crucial role. However, the estimation
accuracy in existing methods remains a challenge, primarily due to the inherent
ambiguity in identifying corresponding areas in adjacent frames for
interpolation. Therefore, enhancing accuracy by distinguishing different
regions before motion estimation is of utmost importance. In this paper, we
introduce a novel solution involving the utilization of open-world segmentation
models, e.g., SAM (Segment Anything Model), to derive Region-Distinguishable
Priors (RDPs) in different frames. These RDPs are represented as
spatial-varying Gaussian mixtures, distinguishing an arbitrary number of areas
with a unified modality. RDPs can be integrated into existing motion-based VFI
methods to enhance features for motion estimation, facilitated by our designed
play-and-plug Hierarchical Region-aware Feature Fusion Module (HRFFM). HRFFM
incorporates RDP into various hierarchical stages of VFI's encoder, using
RDP-guided Feature Normalization (RDPFN) in a residual learning manner. With
HRFFM and RDP, the features within VFI's encoder exhibit similar
representations for matched regions in neighboring frames, thus improving the
synthesis of intermediate frames. Extensive experiments demonstrate that HRFFM
consistently enhances VFI performance across various scenes.

该研究介绍了一种利用开放世界分割模型来提高视频帧插值中运动估计准确性的新方法，通过将区域可区别先验表示为空间变化的高斯混合，设计了可以与现有运动估计方法集成的分层区域感知特征融合模块，实验证明该方法能够在各种场景下显著提升视频帧插值性能。