Traffic accident analysis is pivotal for enhancing public safety and
developing road regulations. Traditional approaches, although widely used, are
often constrained by manual analysis processes, subjective decisions, uni-modal
outputs, as well as privacy issues related to sensitive data. This paper
introduces the idea of AccidentGPT, a foundation model of traffic accident
analysis, which incorporates multi-modal input data to automatically
reconstruct the accident process video with dynamics details, and furthermore
provide multi-task analysis with multi-modal outputs. The design of the
AccidentGPT is empowered with a multi-modality prompt with feedback for
task-oriented adaptability, a hybrid training schema to leverage labelled and
unlabelled data, and a edge-cloud split configuration for data privacy. To
fully realize the functionalities of this model, we proposes several research
opportunities. This paper serves as the stepping stone to fill the gaps in
traditional approaches of traffic accident analysis and attract the research
community attention for automatic, objective, and privacy-preserving traffic
accident analysis.

交通事故分析是提升公共安全和制定道路规章的关键。本文介绍了 AccidentGPT 的概念，它是一个交通事故分析的基础模型，可以通过多模态输入数据自动重建事故过程视频，并提供多任务分析和多模态输出。AccidentGPT 的设计包括多模态输入、任务导向的自适应性、标记和未标记数据的混合训练模式以及边缘 - 云拆分配置，以实现数据隐私保护。为了充分实现此模型的功能，我们提出了一些研究机会。本文作为填补传统交通事故分析方法的空白和引起研究界对自动、客观和隐私保护的交通事故分析的关注的基石。

事故 GPT：用于交通事故分析的大型多模态基础模型

AccidentGPT: Large Multi-Modal Foundation Model for Traffic Accident  Analysis

Jointly harnessing complementary features of multi-modal input data in a
common latent space has been found to be beneficial long ago. However, the
influence of each modality on the models decision remains a puzzle. This study
proposes a deep learning framework for the modality-level interpretation of
multimodal earth observation data in an end-to-end fashion. While leveraging an
explainable machine learning method, namely Occlusion Sensitivity, the proposed
framework investigates the influence of modalities under an early-fusion
scenario in which the modalities are fused before the learning process. We show
that the task of wilderness mapping largely benefits from auxiliary data such
as land cover and night time light data.

本研究提出了一种深度学习框架，用于对多模式地球观测数据进行模态级别的解释。该框架利用可解释的机器学习方法，即遮挡敏感性，研究了模态在学习过程之前融合的情况下对模型决策的影响，并证明了荒野映射任务受到土地覆盖和夜间光数据等辅助数据的极大益处。