We present MM-AU, a novel dataset for Multi-Modal Accident video
Understanding. MM-AU contains 11,727 in-the-wild ego-view accident videos, each
with temporally aligned text descriptions. We annotate over 2.23 million object
boxes and 58,650 pairs of video-based accident reasons, covering 58 accident
categories. MM-AU supports various accident understanding tasks, particularly
multimodal video diffusion to understand accident cause-effect chains for safe
driving. With MM-AU, we present an Abductive accident Video understanding
framework for Safe Driving perception (AdVersa-SD). AdVersa-SD performs video
diffusion via an Object-Centric Video Diffusion (OAVD) method which is driven
by an abductive CLIP model. This model involves a contrastive interaction loss
to learn the pair co-occurrence of normal, near-accident, accident frames with
the corresponding text descriptions, such as accident reasons, prevention
advice, and accident categories. OAVD enforces the causal region learning while
fixing the content of the original frame background in video generation, to
find the dominant cause-effect chain for certain accidents. Extensive
experiments verify the abductive ability of AdVersa-SD and the superiority of
OAVD against the state-of-the-art diffusion models. Additionally, we provide
careful benchmark evaluations for object detection and accident reason
answering since AdVersa-SD relies on precise object and accident reason
information.

MM-AU 是一个多模态事故视频理解的新数据集，包含 11,727 个野外自视角事故视频，每个视频都有与其时间对齐的文本描述。我们注释了超过 2.23 百万个物体框和 58,650 个基于视频的事故原因对，涵盖了 58 个事故类别。MM-AU 支持各种事故理解任务，特别是多模态视频扩散，以理解安全驾驶的事故因果链。通过 MM-AU，我们提出了一种安全驾驶感知的 Abductive 事故视频理解框架 (AdVersa-SD)。AdVersa-SD 通过一种由 Abductive CLIP 模型驱动的对象中心视频扩散 (OAVD) 方法进行视频扩散。该模型通过对正常、几乎事故和事故帧与相应文本描述 (如事故原因、预防建议和事故类别) 的成对共现进行对比交互损失学习，以实现对原始帧背景内容进行修复的视频生成中强制实施因果区域学习，以找到特定事故的主要因果链。大量实验证实了 AdVersa-SD 的推理能力以及 OAVD 相对于最先进的扩散模型的优越性。此外，由于 AdVersa-SD 依赖于精确的物体和事故原因信息，我们还提供了仔细的物体检测和事故原因回答基准评估。