This paper addresses the challenge of developing a robust audio-visual
deepfake detection model. In practical use cases, new generation algorithms are
continually emerging, and these algorithms are not encountered during the
development of detection methods. This calls for the generalization ability of
the method. Additionally, to ensure the credibility of detection methods, it is
beneficial for the model to interpret which cues from the video indicate it is
fake. Motivated by these considerations, we then propose a multi-stream fusion
approach with one-class learning as a representation-level regularization
technique. We study the generalization problem of audio-visual deepfake
detection by creating a new benchmark by extending and re-splitting the
existing FakeAVCeleb dataset. The benchmark contains four categories of fake
video(Real Audio-Fake Visual, Fake Audio-Fake Visual, Fake Audio-Real Visual,
and unsynchronized video). The experimental results show that our approach
improves the model's detection of unseen attacks by an average of 7.31% across
four test sets, compared to the baseline model. Additionally, our proposed
framework offers interpretability, indicating which modality the model
identifies as fake.

该研究针对深度伪造音视频的检测模型的挑战，提出了一种具有多模态融合和一类学习表示级正则化技术的方法，以解决方法的泛化问题和确保模型的可解释性。实验证明该方法对未见攻击的检测平均改进了 7.31%，同时提供了模型对伪造模态的识别结果。

一种基于多流融合和单类学习的音视频深伪造检测方法

A Multi-Stream Fusion Approach with One-Class Learning for Audio-Visual  Deepfake Detection

Trajectory prediction is a fundamental problem and challenge for autonomous
vehicles. Early works mainly focused on designing complicated architectures for
deep-learning-based prediction models in normal-illumination environments,
which fail in dealing with low-light conditions. This paper proposes a novel
approach for trajectory prediction in low-illumination scenarios by leveraging
multi-stream information fusion, which flexibly integrates image, optical flow,
and object trajectory information. The image channel employs Convolutional
Neural Network (CNN) and Long Short-term Memory (LSTM) networks to extract
temporal information from the camera. The optical flow channel is applied to
capture the pattern of relative motion between adjacent camera frames and
modelled by Spatial-Temporal Graph Convolutional Network (ST-GCN). The
trajectory channel is used to recognize high-level interactions between
vehicles. Finally, information from all the three channels is effectively fused
in the prediction module to generate future trajectories of surrounding
vehicles in low-illumination conditions. The proposed multi-channel graph
convolutional approach is validated on HEV-I and newly generated Dark-HEV-I,
egocentric vision datasets that primarily focus on urban intersection
scenarios. The results demonstrate that our method outperforms the baselines,
in standard and low-illumination scenarios. Additionally, our approach is
generic and applicable to scenarios with different types of perception data.
The source code of the proposed approach is available at
this https URL}{this https URL.

本文提出了一种多通道图卷积网络的方法来处理低光照环境下的自动驾驶车辆轨迹预测问题，并在城市路口场景的数据集上进行了验证。