The emergence of artificial intelligence-generated content (AIGC) has raised
concerns about the authenticity of multimedia content in various fields.
However, existing research for forgery content detection has focused mainly on
binary classification tasks of complete videos, which has limited applicability
in industrial settings. To address this gap, we propose UMMAFormer, a novel
universal transformer framework for temporal forgery localization (TFL) that
predicts forgery segments with multimodal adaptation. Our approach introduces a
Temporal Feature Abnormal Attention (TFAA) module based on temporal feature
reconstruction to enhance the detection of temporal differences. We also design
a Parallel Cross-Attention Feature Pyramid Network (PCA-FPN) to optimize the
Feature Pyramid Network (FPN) for subtle feature enhancement. To evaluate the
proposed method, we contribute a novel Temporal Video Inpainting Localization
(TVIL) dataset specifically tailored for video inpainting scenes. Our
experiments show that our approach achieves state-of-the-art performance on
benchmark datasets, including Lav-DF, TVIL, and Psynd, significantly
outperforming previous methods. The code and data are available at
this https URL

提出 UMMAFormer，一种新颖的通用 Transformer 框架，用于时序伪造定位，通过多模态适应预测伪造段，包括引入基于时序特征重建的时序特征异常关注模块（TFAA）以增强时序差异的检测，设计用于微妙特征增强的平行交叉关注特征金字塔网络（PCA-FPN），并构建适用于视频修复场景的新型时序视频修复定位（TVIL）数据集，实验结果表明我们的方法在众多基准数据集上取得了最优性能。

UMMAFormer：一种用于时间伪造定位的通用多模式自适应 Transformer 框架

UMMAFormer: A Universal Multimodal-adaptive Transformer Framework for  Temporal Forgery Localization

The rapid progress of photorealistic synthesis techniques has reached at a
critical point where the boundary between real and manipulated images starts to
blur. Thus, benchmarking and advancing digital forgery analysis have become a
pressing issue. However, existing face forgery datasets either have limited
diversity or only support coarse-grained analysis. To counter this emerging
threat, we construct the ForgeryNet dataset, an extremely large face forgery
dataset with unified annotations in image- and video-level data across four
tasks: 1) Image Forgery Classification, including two-way (real / fake),
three-way (real / fake with identity-replaced forgery approaches / fake with
identity-remained forgery approaches), and n-way (real and 15 respective
forgery approaches) classification. 2) Spatial Forgery Localization, which
segments the manipulated area of fake images compared to their corresponding
source real images. 3) Video Forgery Classification, which re-defines the
video-level forgery classification with manipulated frames in random positions.
This task is important because attackers in real world are free to manipulate
any target frame. and 4) Temporal Forgery Localization, to localize the
temporal segments which are manipulated. ForgeryNet is by far the largest
publicly available deep face forgery dataset in terms of data-scale (2.9
million images, 221,247 videos), manipulations (7 image-level approaches, 8
video-level approaches), perturbations (36 independent and more mixed
perturbations) and annotations (6.3 million classification labels, 2.9 million
manipulated area annotations and 221,247 temporal forgery segment labels). We
perform extensive benchmarking and studies of existing face forensics methods
and obtain several valuable observations.

构建了最大的公共面部伪造数据集 ForgeryNet，涉及 2.9 百万张图片和 221,247 个视频，在图像和视频级数据上包含 4 个任务：图像伪造分类、空间伪造定位、视频伪造分类、和时间伪造定位，提供了对面部鉴定算法的完整评价。