The proliferation of deepfake faces poses huge potential negative impacts on our daily lives. Despite substantial advancements in deepfake detection over these years, the generalizability of existing methods against forgeries from unseen datasets or created by emerging generative models remains constrained. In this paper, inspired by the zero-shot advantages of Vision-Language Models (VLMs), we propose a novel approach that repurposes a well-trained VLM for general deepfake detection. Motivated by the model reprogramming paradigm that manipulates the model prediction via data perturbations, our method can reprogram a pretrained VLM model (e.g., CLIP) solely based on manipulating its input without tuning the inner parameters. Furthermore, we insert a pseudo-word guided by facial identity into the text prompt. Extensive experiments on several popular benchmarks demonstrate that (1) the cross-dataset and cross-manipulation performances of deepfake detection can be significantly and consistently improved (e.g., over 88% AUC in cross-dataset setting from FF++ to WildDeepfake) using a pre-trained CLIP model with our proposed reprogramming method; (2) our superior performances are at less cost of trainable parameters, making it a promising approach for real-world applications.

本研究针对现有深度伪造检测方法在未见数据集和新兴生成模型上泛化能力不足的问题，提出了一种新颖的方法，利用视觉-语言模型（VLM）的零-shot优势，实现通用深度伪造检测。通过简单操控输入而不调整内部参数，本方法显著提高了跨数据集和跨操控的深度伪造检测性能，具有良好的实际应用潜力。

站在巨人的肩膀上：重编视觉-语言模型进行通用深度伪造检测