The widespread use of diffusion methods enables the creation of highly
realistic images on demand, thereby posing significant risks to the integrity
and safety of online information and highlighting the necessity of DeepFake
detection. Our analysis of features extracted by traditional image encoders
reveals that both low-level and high-level features offer distinct advantages
in identifying DeepFake images produced by various diffusion methods. Inspired
by this finding, we aim to develop an effective representation that captures
both low-level and high-level features to detect diffusion-based DeepFakes. To
address the problem, we propose a text modality-oriented feature extraction
method, termed TOFE. Specifically, for a given target image, the representation
we discovered is a corresponding text embedding that can guide the generation
of the target image with a specific text-to-image model. Experiments conducted
across ten diffusion types demonstrate the efficacy of our proposed method.

通过传统图像编码器提取的特征分析发现，无论是低级特征还是高级特征都对于识别不同扩散方法制作的 DeepFake 图像具有独特的优势。受到这一发现的启发，我们旨在开发一种有效的表示方法，以捕捉低级和高级特征来检测基于扩散的 DeepFakes。为了解决这个问题，我们提出了一种文本模态导向的特征提取方法，称为 TOFE。具体而言，对于给定的目标图像，我们发现的表示是一个相应的文本嵌入，可引导具有特定文本到图像模型的目标图像生成。在十种扩散类型的实验中证明了我们提出的方法的功效。