Recent advancements in diffusion-based video generation have showcased
remarkable results, yet the gap between synthetic and real-world videos remains
under-explored. In this study, we examine this gap from three fundamental
perspectives: appearance, motion, and geometry, comparing real-world videos
with those generated by a state-of-the-art AI model, Stable Video Diffusion. To
achieve this, we train three classifiers using 3D convolutional networks, each
targeting distinct aspects: vision foundation model features for appearance,
optical flow for motion, and monocular depth for geometry. Each classifier
exhibits strong performance in fake video detection, both qualitatively and
quantitatively. This indicates that AI-generated videos are still easily
detectable, and a significant gap between real and fake videos persists.
Furthermore, utilizing the Grad-CAM, we pinpoint systematic failures of
AI-generated videos in appearance, motion, and geometry. Finally, we propose an
Ensemble-of-Experts model that integrates appearance, optical flow, and depth
information for fake video detection, resulting in enhanced robustness and
generalization ability. Our model is capable of detecting videos generated by
Sora with high accuracy, even without exposure to any Sora videos during
training. This suggests that the gap between real and fake videos can be
generalized across various video generative models. Project page:
this https URL

通过实验分析外观、运动和几何等方面，在外观、运动和几何等方面对比了真实世界的视频和由先进 AI 模型生成的视频，揭示了真实世界视频与合成视频之间的差距，并提出了集成外观、光流和深度信息的专家集合模型以进行假视频检测，提高了鲁棒性和泛化能力。

检测像 Sora 这样的 AI 生成视频中的关键内容

What Matters in Detecting AI-Generated Videos like Sora?

With the rapid advancement in video generation, people can conveniently
utilize video generation models to create videos tailored to their specific
desires. Nevertheless, there are also growing concerns about their potential
misuse in creating and disseminating false information.
In this work, we introduce VGMShield: a set of three straightforward but
pioneering mitigations through the lifecycle of fake video generation. We start
from \textit{fake video detection} trying to understand whether there is
uniqueness in generated videos and whether we can differentiate them from real
videos; then, we investigate the \textit{tracing} problem, which maps a fake
video back to a model that generates it. Towards these, we propose to leverage
pre-trained models that focus on {\it spatial-temporal dynamics} as the
backbone to identify inconsistencies in videos. Through experiments on seven
state-of-the-art open-source models, we demonstrate that current models still
cannot perfectly handle spatial-temporal relationships, and thus, we can
accomplish detection and tracing with nearly perfect accuracy.
Furthermore, anticipating future generative model improvements, we propose a
{\it prevention} method that adds invisible perturbations to images to make the
generated videos look unreal. Together with fake video detection and tracing,
our multi-faceted set of solutions can effectively mitigate misuse of video
generative models.

VGMShield introduces three straightforward but pioneering mitigations for fake video generation: fake video detection, tracing, and prevention via leveraging spatial-temporal dynamics and invisible perturbations, achieving nearly perfect accuracy in detection and tracing.

VGMShield: 缓解视频生成模型的误用

VGMShield: Mitigating Misuse of Video Generative Models

Recently, image manipulation has achieved rapid growth due to the advancement
of sophisticated image editing tools. A recent surge of generated fake imagery
and videos using neural networks is DeepFake. DeepFake algorithms can create
fake images and videos that humans cannot distinguish from authentic ones.
(GANs) have been extensively used for creating realistic images without
accessing the original images. Therefore, it is become essential to detect fake
videos to avoid spreading false information. This paper presents a survey of
methods used to detect DeepFakes and datasets available for detecting DeepFakes
in the literature to date. We present extensive discussions and research trends
related to DeepFake technologies.

本文简要介绍了图像处理、深度伪造技术以及用于检测这些技术的方法和数据集。

审判法院深度伪造检测综述

A Survey of Deep Fake Detection for Trial Courts

With recent advances in computer vision and graphics, it is now possible to
generate videos with extremely realistic synthetic faces, even in real time.
Countless applications are possible, some of which raise a legitimate alarm,
calling for reliable detectors of fake videos. In fact, distinguishing between
original and manipulated video can be a challenge for humans and computers
alike, especially when the videos are compressed or have low resolution, as it
often happens on social networks. Research on the detection of face
manipulations has been seriously hampered by the lack of adequate datasets. To
this end, we introduce a novel face manipulation dataset of about half a
million edited images (from over 1000 videos). The manipulations have been
generated with a state-of-the-art face editing approach. It exceeds all
existing video manipulation datasets by at least an order of magnitude. Using
our new dataset, we introduce benchmarks for classical image forensic tasks,
including classification and segmentation, considering videos compressed at
various quality levels. In addition, we introduce a benchmark evaluation for
creating indistinguishable forgeries with known ground truth; for instance with
generative refinement models.

通过引入新的半百万级别的面部操作数据集，本文提出了对诸如经典图像取证分类、分割、以及众多优化方法进行基准测试、鉴定的处理任务，包括压缩不同质量级别下的视频，并达到超越现有所有视频操纵数据集一个数量级的状态，旨在通过根据目标制作难以区分的赝品来进行基准评估。