Recent advancements in diffusion-based video generation have showcased
remarkable results, yet the gap between synthetic and real-world videos remains
under-explored. In this study, we examine this gap from three fundamental
perspectives: appearance, motion, and geometry, comparing real-world videos
with those generated by a state-of-the-art AI model, Stable Video Diffusion. To
achieve this, we train three classifiers using 3D convolutional networks, each
targeting distinct aspects: vision foundation model features for appearance,
optical flow for motion, and monocular depth for geometry. Each classifier
exhibits strong performance in fake video detection, both qualitatively and
quantitatively. This indicates that AI-generated videos are still easily
detectable, and a significant gap between real and fake videos persists.
Furthermore, utilizing the Grad-CAM, we pinpoint systematic failures of
AI-generated videos in appearance, motion, and geometry. Finally, we propose an
Ensemble-of-Experts model that integrates appearance, optical flow, and depth
information for fake video detection, resulting in enhanced robustness and
generalization ability. Our model is capable of detecting videos generated by
Sora with high accuracy, even without exposure to any Sora videos during
training. This suggests that the gap between real and fake videos can be
generalized across various video generative models. Project page:
this https URL

通过实验分析外观、运动和几何等方面，在外观、运动和几何等方面对比了真实世界的视频和由先进 AI 模型生成的视频，揭示了真实世界视频与合成视频之间的差距，并提出了集成外观、光流和深度信息的专家集合模型以进行假视频检测，提高了鲁棒性和泛化能力。