As Deepfake contents continue to proliferate on the internet, advancing face
manipulation forensics has become a pressing issue. To combat this emerging
threat, previous methods mainly focus on studying how to distinguish authentic
and manipulated face images. Despite impressive, image-level classification
lacks explainability and is limited to some specific application scenarios.
Existing forgery localization methods suffer from imprecise and inconsistent
pixel-level annotations. To alleviate these problems, this paper first
re-constructs the FaceForensics++ dataset by introducing pixel-level
annotations, then builds an extensive benchmark for localizing tampered
regions. Next, a novel Multi-Spectral Class Center Network (MSCCNet) is
proposed for face manipulation detection and localization. Specifically,
inspired by the power of frequency-related forgery traces, we design
Multi-Spectral Class Center (MSCC) module to learn more generalizable and
semantic-agnostic features. Based on the features of different frequency bands,
the MSCC module collects multispectral class centers and computes
pixel-to-class relations. Applying multi-spectral class-level representations
suppresses the semantic information of the visual concepts, which is
insensitive to manipulations. Furthermore, we propose a Multi-level Features
Aggregation (MFA) module to employ more low-level forgery artifacts and
structure textures. Experimental results quantitatively and qualitatively
indicate the effectiveness and superiority of the proposed MSCCNet on
comprehensive localization benchmarks. We expect this work to inspire more
studies on pixel-level face manipulation localization. The annotations and code
will be available.

本研究重建了 FaceForensics ++ 数据集，引入像素级别的注释，并提出了一种新的多光谱中心分类网络（MSCCNet）来检测和定位面部操纵。