Deep video inpainting is typically used as malicious manipulation to remove
important objects for creating fake videos. It is significant to identify the
inpainted regions blindly. This letter proposes a simple yet effective forensic
scheme for Video Inpainting LOcalization with ContrAstive Learning (ViLocal).
Specifically, a 3D Uniformer encoder is applied to the video noise residual for
learning effective spatiotemporal forensic features. To enhance the
discriminative power, supervised contrastive learning is adopted to capture the
local inconsistency of inpainted videos through attracting/repelling the
positive/negative pristine and forged pixel pairs. A pixel-wise inpainting
localization map is yielded by a lightweight convolution decoder with a
specialized two-stage training strategy. To prepare enough training samples, we
build a video object segmentation dataset of 2500 videos with pixel-level
annotations per frame. Extensive experimental results validate the superiority
of ViLocal over state-of-the-arts. Code and dataset will be available at
this https URL

基于对比学习的 Video Inpainting LOcalization (ViLocal) 方案使用 3D Uniformer 编码器学习了有效的时空取证特征，通过吸引 / 排斥正 / 负原始和伪造像素对捕捉到了图像修复不一致，然后通过专门的两阶段训练策略利用轻量级卷积解码器生成了一个像素级修复定位地图，实验证明 ViLocal 优于现有技术。

对比学习下的视频修复定位

Video Inpainting Localization with Contrastive Learning

Recently, removing objects from videos and filling in the erased regions
using deep video inpainting (VI) algorithms has attracted considerable
attention. Usually, a video sequence and object segmentation masks for all
frames are required as the input for this task. However, in real-world
applications, providing segmentation masks for all frames is quite difficult
and inefficient. Therefore, we deal with VI in a one-shot manner, which only
takes the initial frame's object mask as its input. Although we can achieve
that using naive combinations of video object segmentation (VOS) and VI
methods, they are sub-optimal and generally cause critical errors. To address
that, we propose a unified pipeline for one-shot video inpainting (OSVI). By
jointly learning mask prediction and video completion in an end-to-end manner,
the results can be optimal for the entire task instead of each separate module.
Additionally, unlike the two stage methods that use the predicted masks as
ground truth cues, our method is more reliable because the predicted masks can
be used as the network's internal guidance. On the synthesized datasets for
OSVI, our proposed method outperforms all others both quantitatively and
qualitatively.

本研究提出了一种统一的方法来实现一次性视频修复，它可以同时进行遮罩预测和视频完成，并通过端到端的学习来优化整个任务，其效果在合成数据集上表现最佳。