There is strong interest in the generation of synthetic video imagery of
people talking for various purposes, including entertainment, communication,
training, and advertisement. With the development of deep fake generation
models, synthetic video imagery will soon be visually indistinguishable to the
naked eye from a naturally capture video. In addition, many methods are
continuing to improve to avoid more careful, forensic visual analysis. Some
deep fake videos are produced through the use of facial puppetry, which
directly controls the head and face of the synthetic image through the
movements of the actor, allow the actor to 'puppet' the image of another. In
this paper, we address the question of whether one person's movements can be
distinguished from the original speaker by controlling the visual appearance of
the speaker but transferring the behavior signals from another source. We
conduct a study by comparing synthetic imagery that: 1) originates from a
different person speaking a different utterance, 2) originates from the same
person speaking a different utterance, and 3) originates from a different
person speaking the same utterance. Our study shows that synthetic videos in
all three cases are seen as less real and less engaging than the original
source video. Our results indicate that there could be a behavioral signature
that is detectable from a person's movements that is separate from their visual
appearance, and that this behavioral signature could be used to distinguish a
deep fake from a properly captured video.

通过比较不同来源的合成视频，本研究发现合成视频在视觉和互动方面与原始视频相比都存在差异，这表明行为特征可以用于识别深度伪造视频。

探测 DeepFake 视频中的行为特征

Study of detecting behavioral signatures within DeepFake videos

Facial manipulation by deep fake has caused major security risks and raised
severe societal concerns. As a countermeasure, a number of deep fake detection
methods have been proposed recently. Most of them model deep fake detection as
a binary classification problem using a backbone convolutional neural network
(CNN) architecture pretrained for the task. These CNN-based methods have
demonstrated very high efficacy in deep fake detection with the Area under the
Curve (AUC) as high as 0.99. However, the performance of these methods degrades
significantly when evaluated across datasets. In this paper, we formulate deep
fake detection as a hybrid combination of supervised and reinforcement learning
(RL) to improve its cross-dataset generalization performance. The proposed
method chooses the top-k augmentations for each test sample by an RL agent in
an image-specific manner. The classification scores, obtained using CNN, of all
the augmentations of each test image are averaged together for final real or
fake classification. Through extensive experimental validation, we demonstrate
the superiority of our method over existing published research in cross-dataset
generalization of deep fake detectors, thus obtaining state-of-the-art
performance.

本研究使用深度强化学习与卷积神经网络相结合，设计了一种新的深度伪造检测算法，通过统计每张测试图像的各种增强方法的分类分数来检测深度伪造的图片，实验证明该方法在跨数据集的泛化性能方面优于现有的模型，并获得最先进的性能。