Existing face forgery detection models try to discriminate fake images by
detecting only spatial artifacts (e.g., generative artifacts, blending) or
mainly temporal artifacts (e.g., flickering, discontinuity). They may
experience significant performance degradation when facing out-domain
artifacts. In this paper, we propose to capture both spatial and temporal
artifacts in one model for face forgery detection. A simple idea is to leverage
a spatiotemporal model (3D ConvNet). However, we find that it may easily rely
on one type of artifact and ignore the other. To address this issue, we present
a novel training strategy called AltFreezing for more general face forgery
detection. The AltFreezing aims to encourage the model to detect both spatial
and temporal artifacts. It divides the weights of a spatiotemporal network into
two groups: spatial-related and temporal-related. Then the two groups of
weights are alternately frozen during the training process so that the model
can learn spatial and temporal features to distinguish real or fake videos.
Furthermore, we introduce various video-level data augmentation methods to
improve the generalization capability of the forgery detection model. Extensive
experiments show that our framework outperforms existing methods in terms of
generalization to unseen manipulations and datasets. Code is available at
https: //github.com/ZhendongWang6/AltFreezing.

本文提出了一种用于人脸伪造检测的深度学习模型，能够同时检测空域和时域伪像，在训练过程中分别学习这两种伪像的特征，结合了多个视频层级数据增强方法，可提高模型泛化性能，并在未见过的操作和数据集上显著优于现有方法。