The task of video inpainting detection is to expose the pixel-level inpainted
regions within a video sequence. Existing methods usually focus on leveraging
spatial and temporal inconsistencies. However, these methods typically employ
fixed operations to combine spatial and temporal clues, limiting their
applicability in different scenarios. In this paper, we introduce a novel
Multilateral Temporal-view Pyramid Transformer ({\em MumPy}) that collaborates
spatial-temporal clues flexibly. Our method utilizes a newly designed
multilateral temporal-view encoder to extract various collaborations of
spatial-temporal clues and introduces a deformable window-based temporal-view
interaction module to enhance the diversity of these collaborations.
Subsequently, we develop a multi-pyramid decoder to aggregate the various types
of features and generate detection maps. By adjusting the contribution strength
of spatial and temporal clues, our method can effectively identify inpainted
regions. We validate our method on existing datasets and also introduce a new
challenging and large-scale Video Inpainting dataset based on the YouTube-VOS
dataset, which employs several more recent inpainting methods. The results
demonstrate the superiority of our method in both in-domain and cross-domain
evaluation scenarios.

本文介绍了一种名为 MumPy 的新型多边时空视图金字塔变换器，提出了一种利用新设计的多边时空视图编码器和可变形窗口时空视图交互模块的方法来灵活地合作处理空间 - 时间线索，以及开发了一个多金字塔解码器来聚合各种特征并生成检测图，通过调整空间和时间线索的贡献强度，我们的方法能够有效地识别修复区域。我们在现有数据集上验证了我们的方法，并且基于 YouTube-VOS 数据集引入了一个新的具有挑战性和大规模的视频修复数据集，该数据集使用了几种更近期的修复方法。结果显示了我们的方法在域内和跨域评估场景中的优越性。