With the rise of deep learning, generative models have enabled the creation
of highly realistic synthetic images, presenting challenges due to their
potential misuse. While research in Deepfake detection has grown rapidly in
response, many detection methods struggle with unseen Deepfakes generated by
new synthesis techniques. To address this generalisation challenge, we propose
a novel Deepfake detection approach by adapting rich information encoded inside
the Foundation Models with rich information encoded inside, specifically using
the image encoder from CLIP which has demonstrated strong zero-shot capability
for downstream tasks. Inspired by the recent advances of parameter efficient
fine-tuning, we propose a novel side-network-based decoder to extract spatial
and temporal cues from the given video clip, with the promotion of the Facial
Component Guidance (FCG) to guidencourage the spatial feature to include
features of key facial parts for more robust and general Deepfake detection.
Through extensive cross-dataset evaluations, our approach exhibits superior
effectiveness in identifying unseen Deepfake samples, achieving notable
performance improvementsuccess even with limited training samples and
manipulation types. Our model secures an average performance enhancement of
0.9% AUROC in cross-dataset assessments comparing with state-of-the-art
methods, especiallytablishing a significant lead of achieving 4.4% improvement
on the challenging DFDC dataset.

通过在 Foundation Models 中加入富信息，利用了 CLIP 的图像编码器和 Facial Component Guidance 以提高 Deepfake 检测的鲁棒性和泛化性能。在跨数据集的评估中，我们的方法在识别未见过的 Deepfake 样本方面表现出卓越的有效性，并取得了显著的性能提升。

基于面部特征引导适应的视频深度伪造检测的进一步普适化

Towards More General Video-based Deepfake Detection through Facial  Feature Guided Adaptation for Foundation Model

Learning-based methods are believed to work well for unconstrained gaze
estimation, i.e. gaze estimation from a monocular RGB camera without
assumptions regarding user, environment, or camera. However, current gaze
datasets were collected under laboratory conditions and methods were not
evaluated across multiple datasets. Our work makes three contributions towards
addressing these limitations. First, we present the MPIIGaze that contains
213,659 full face images and corresponding ground-truth gaze positions
collected from 15 users during everyday laptop use over several months. An
experience sampling approach ensured continuous gaze and head poses and
realistic variation in eye appearance and illumination. To facilitate
cross-dataset evaluations, 37,667 images were manually annotated with eye
corners, mouth corners, and pupil centres. Second, we present an extensive
evaluation of state-of-the-art gaze estimation methods on three current
datasets, including MPIIGaze. We study key challenges including target gaze
range, illumination conditions, and facial appearance variation. We show that
image resolution and the use of both eyes affect gaze estimation performance
while head pose and pupil centre information are less informative. Finally, we
propose GazeNet, the first deep appearance-based gaze estimation method.
GazeNet improves the state of the art by 22% percent (from a mean error of 13.9
degrees to 10.8 degrees) for the most challenging cross-dataset evaluation.

我们提出了 MPIIGaze，包括 213659 个人的实验数据，并对现有的三个数据集进行了广泛的评估，研究了目标视线范围、光照条件和面部外观变化等关键挑战，提出了第一种深度外观估计方法 GazeNet，并将平均误差从 13.9 度提高到 10.8 度，这是当前技术水平的改进。