Free-form video inpainting is a very challenging task that could be widely
used for video editing such as text removal. Existing patch-based methods could
not handle non-repetitive structures such as faces, while directly applying
image-based inpainting models to videos will result in temporal inconsistency
(see this http URL ). In this paper, we introduce a deep learn-ing
based free-form video inpainting model, with proposed 3D gated convolutions to
tackle the uncertainty of free-form masks and a novel Temporal PatchGAN loss to
enhance temporal consistency. In addition, we collect videos and design a
free-form mask generation algorithm to build the free-form video inpainting
(FVI) dataset for training and evaluation of video inpainting models. We
demonstrate the benefits of these components and experiments on both the
FaceForensics and our FVI dataset suggest that our method is superior to
existing ones. Related source code, full-resolution result videos and the FVI
dataset could be found on Github
this https URL .

本研究提出了一种基于深度学习的自由形式视频修复模型，采用 3D 时域门控卷积和新颖的时间 PatchGAN loss 解决非重复的结构物体遮挡及时间上的一致性问题，并通过收集视频并设计自由形式掩膜生成算法来构建 FVI 数据集，该模型被证明优于现有方法，特别适用于文字去除等视频编辑方向。