Significant advancements made in the generation of deepfakes have caused
security and privacy issues. Attackers can easily impersonate a person's
identity in an image by replacing his face with the target person's face.
Moreover, a new domain of cloning human voices using deep-learning technologies
is also emerging. Now, an attacker can generate realistic cloned voices of
humans using only a few seconds of audio of the target person. With the
emerging threat of potential harm deepfakes can cause, researchers have
proposed deepfake detection methods. However, they only focus on detecting a
single modality, i.e., either video or audio. On the other hand, to develop a
good deepfake detector that can cope with the recent advancements in deepfake
generation, we need to have a detector that can detect deepfakes of multiple
modalities, i.e., videos and audios. To build such a detector, we need a
dataset that contains video and respective audio deepfakes. We were able to
find a most recent deepfake dataset, Audio-Video Multimodal Deepfake Detection
Dataset (FakeAVCeleb), that contains not only deepfake videos but synthesized
fake audios as well. We used this multimodal deepfake dataset and performed
detailed baseline experiments using state-of-the-art unimodal, ensemble-based,
and multimodal detection methods to evaluate it. We conclude through detailed
experimentation that unimodals, addressing only a single modality, video or
audio, do not perform well compared to ensemble-based methods. Whereas purely
multimodal-based baselines provide the worst performance.

该篇研究通过使用最新的深度伪造音频和视频数据集（FakeAVCeleb）进行了详细的基准实验，得出结论：单模态深度伪造检测方法仅仅能够适应单个媒体类型，仅使用视频或音频来检测不够理想，最佳的检测方法是集成式检测方法。