The rapid advancement in text-to-video (T2V) generative models has enabled
the synthesis of high-fidelity video content guided by textual descriptions.
Despite this significant progress, these models are often susceptible to
hallucination, generating contents that contradict the input text, which poses
a challenge to their reliability and practical deployment. To address this
critical issue, we introduce the SoraDetector, a novel unified framework
designed to detect hallucinations across diverse large T2V models, including
the cutting-edge Sora model. Our framework is built upon a comprehensive
analysis of hallucination phenomena, categorizing them based on their
manifestation in the video content. Leveraging the state-of-the-art keyframe
extraction techniques and multimodal large language models, SoraDetector first
evaluates the consistency between extracted video content summary and textual
prompts, then constructs static and dynamic knowledge graphs (KGs) from frames
to detect hallucination both in single frames and across frames. Sora Detector
provides a robust and quantifiable measure of consistency, static and dynamic
hallucination. In addition, we have developed the Sora Detector Agent to
automate the hallucination detection process and generate a complete video
quality report for each input video. Lastly, we present a novel meta-evaluation
benchmark, T2VHaluBench, meticulously crafted to facilitate the evaluation of
advancements in T2V hallucination detection. Through extensive experiments on
videos generated by Sora and other large T2V models, we demonstrate the
efficacy of our approach in accurately detecting hallucinations. The code and
dataset can be accessed via GitHub.

为了解决文本到视频（T2V）生成模型中的幻觉问题，本研究引入了一种名为 SoraDetector 的新型统一框架，通过对幻觉现象的综合分析和关键帧提取技术，利用多模式大型语言模型构建静态和动态知识图，在单帧和多帧中检测幻觉，并通过自动化工具生成完整的视频质量报告。同时，还开发了一个用于评估 T2V 幻觉检测进展的元评估基准 T2VHaluBench，并通过在 Sora 和其他大型 T2V 模型生成的视频上进行广泛实验，证明了我们的方法在准确检测幻觉方面的有效性。