The task of deepfakes detection is far from being solved by speech or vision
researchers. Several publicly available databases of fake synthetic video and
speech were built to aid the development of detection methods. However,
existing databases typically focus on visual or voice modalities and provide no
proof that their deepfakes can in fact impersonate any real person. In this
paper, we present the first realistic audio-visual database of deepfakes
SWAN-DF, where lips and speech are well synchronized and video have high visual
and audio qualities. We took the publicly available SWAN dataset of real videos
with different identities to create audio-visual deepfakes using several models
from DeepFaceLab and blending techniques for face swapping and HiFiVC, DiffVC,
YourTTS, and FreeVC models for voice conversion. From the publicly available
speech dataset LibriTTS, we also created a separate database of only audio
deepfakes LibriTTS-DF using several latest text to speech methods: YourTTS,
Adaspeech, and TorToiSe. We demonstrate the vulnerability of a state of the art
speaker recognition system, such as ECAPA-TDNN-based model from SpeechBrain, to
the synthetic voices. Similarly, we tested face recognition system based on the
MobileFaceNet architecture to several variants of our visual deepfakes. The
vulnerability assessment show that by tuning the existing pretrained deepfake
models to specific identities, one can successfully spoof the face and speaker
recognition systems in more than 90% of the time and achieve a very realistic
looking and sounding fake video of a given person.

该研究论文介绍了首个真实的音频 - 视觉深度伪造数据库 SWAN-DF，其中嘴唇和语音同步，并具有高质量的视觉和音频。研究表明，通过调整现有的预训练深度伪造模型来适应特定身份，可以在超过 90% 的时间内成功欺骗人脸和说话人识别系统，并获得非常逼真的伪造视频。