Since photorealistic faces can be readily generated by facial manipulation
technologies nowadays, potential malicious abuse of these technologies has
drawn great concerns. Numerous deepfake detection methods are thus proposed.
However, existing methods only focus on detecting one-step facial manipulation.
As the emergence of easy-accessible facial editing applications, people can
easily manipulate facial components using multi-step operations in a sequential
manner. This new threat requires us to detect a sequence of facial
manipulations, which is vital for both detecting deepfake media and recovering
original faces afterwards. Motivated by this observation, we emphasize the need
and propose a novel research problem called Detecting Sequential DeepFake
Manipulation (Seq-DeepFake). Unlike the existing deepfake detection task only
demanding a binary label prediction, detecting Seq-DeepFake manipulation
requires correctly predicting a sequential vector of facial manipulation
operations. To support a large-scale investigation, we construct the first
Seq-DeepFake dataset, where face images are manipulated sequentially with
corresponding annotations of sequential facial manipulation vectors. Based on
this new dataset, we cast detecting Seq-DeepFake manipulation as a specific
image-to-sequence (e.g. image captioning) task and propose a concise yet
effective Seq-DeepFake Transformer (SeqFakeFormer). Moreover, we build a
comprehensive benchmark and set up rigorous evaluation protocols and metrics
for this new research problem. Extensive experiments demonstrate the
effectiveness of SeqFakeFormer. Several valuable observations are also revealed
to facilitate future research in broader deepfake detection problems.

文章重点提出了一个名为 Seq-DeepFake 的新颖研究问题，即在检测 deepfake 媒体和恢复原始面孔之后，需要检测一系列面部操作。通过构建第一个 Seq-DeepFake 数据集，并将其作为一种特定的图像到序列任务，提出了一个简洁有效的 Seq-DeepFake Transformer 模型，并建立了全面的评估方法。实验证明了该模型的有效性。