The proliferation of social media platforms such as Twitter, Instagram, and
Weibo has significantly enhanced the dissemination of false information. This
phenomenon grants both individuals and governmental entities the ability to
shape public opinions, highlighting the need for deploying effective detection
methods. In this paper, we propose GraMuFeN, a model designed to detect fake
content by analyzing both the textual and image content of news. GraMuFeN
comprises two primary components: a text encoder and an image encoder. For
textual analysis, GraMuFeN treats each text as a graph and employs a Graph
Convolutional Neural Network (GCN) as the text encoder. Additionally, the
pre-trained ResNet-152, as a Convolutional Neural Network (CNN), has been
utilized as the image encoder. By integrating the outputs from these two
encoders and implementing a contrastive similarity loss function, GraMuFeN
achieves remarkable results. Extensive evaluations conducted on two publicly
available benchmark datasets for social media news indicate a 10 % increase in
micro F1-Score, signifying improvement over existing state-of-the-art models.
These findings underscore the effectiveness of combining GCN and CNN models for
detecting fake news in multi-modal data, all while minimizing the additional
computational burden imposed by model parameters.

通过分析新闻的文本和图像内容，本文提出了 GraMuFeN 模型，结合了图卷积神经网络和卷积神经网络的输出，并应用对比相似度损失函数，取得了显著的结果，用于检测社交媒体中的虚假消息。

GRaMuFeN: 基于图的社交媒体多模态虚假新闻检测

GRaMuFeN: Graph-based Multi-modal Fake News Detection in Social Media

Deepfakes are the synthesized digital media in order to create
ultra-realistic fake videos to trick the spectator. Deep generative algorithms,
such as, Generative Adversarial Networks(GAN) are widely used to accomplish
such tasks. This approach synthesizes pseudo-realistic contents that are very
difficult to distinguish by traditional detection methods. In most cases,
Convolutional Neural Network(CNN) based discriminators are being used for
detecting such synthesized media. However, it emphasise primarily on the
spatial attributes of individual video frames, thereby fail to learn the
temporal information from their inter-frame relations. In this paper, we
leveraged an optical flow based feature extraction approach to extract the
temporal features, which are then fed to a hybrid model for classification.
This hybrid model is based on the combination of CNN and recurrent neural
network (RNN) architectures. The hybrid model provides effective performance on
open source data-sets such as, DFDC, FF++ and Celeb-DF. This proposed method
shows an accuracy of 66.26%, 91.21% and 79.49% in DFDC, FF++, and Celeb-DF
respectively with a very reduced No of sample size of approx 100
samples(frames). This promises early detection of fake contents compared to
existing modalities.

本文主要研究深度伪造影像技术中因时域信息缺失导致传统方法检测失误的问题，并基于光流特征提取、CNN 和 RNN 混合模型开展研究，实现对 DFDC、FF++ 和 Celeb-DF 等开源数据集中深度伪造影像的有效检测，样本数量不足 100 帧时在 DFDC、FF++、Celeb-DF 数据集上的精度为 66.26%、91.21% 和 79.49%，有望实现对深度伪造内容的早期检测。

基于光流特征的混合卷积神经网络 - 长短期记忆网络在视频深度伪造检测中的应用

A Hybrid CNN-LSTM model for Video Deepfake Detection by Leveraging Optical Flow Features

The recent proliferation of fake portrait videos poses direct threats on
society, law, and privacy. Believing the fake video of a politician,
distributing fake pornographic content of celebrities, fabricating impersonated
fake videos as evidence in courts are just a few real world consequences of
deep fakes. We present a novel approach to detect synthetic content in portrait
videos, as a preventive solution for the emerging threat of deep fakes. In
other words, we introduce a deep fake detector. We observe that detectors
blindly utilizing deep learning are not effective in catching fake content, as
generative models produce formidably realistic results. Our key assertion
follows that biological signals hidden in portrait videos can be used as an
implicit descriptor of authenticity, because they are neither spatially nor
temporally preserved in fake content. To prove and exploit this assertion, we
first engage several signal transformations for the pairwise separation
problem, achieving 99.39% accuracy. Second, we utilize those findings to
formulate a generalized classifier for fake content, by analyzing proposed
signal transformations and corresponding feature sets. Third, we generate novel
signal maps and employ a CNN to improve our traditional classifier for
detecting synthetic content. Lastly, we release an "in the wild" dataset of
fake portrait videos that we collected as a part of our evaluation process. We
evaluate FakeCatcher on several datasets, resulting with 96%, 94.65%, 91.50%,
and 91.07% accuracies, on Face Forensics, Face Forensics++, CelebDF, and on our
new Deep Fakes Dataset respectively. We also analyze signals from various
facial regions, under image distortions, with varying segment durations, from
different generators, against unseen datasets, and under several dimensionality
reduction techniques.

本文提出了一种新颖的方法来检测肖像视频的合成内容，通过使用隐藏在肖像视频中的生物信号作为真实性的隐含描述符，并通过信号转化和卷积神经网络，最终实现了检测 deep fake 的效果达到了高准确率。