Traffic accidents frequently lead to fatal injuries, contributing to over 50
million deaths until 2023. To mitigate driving hazards and ensure personal
safety, it is crucial to assist vehicles in anticipating important objects
during travel. Previous research on important object detection primarily
assessed the importance of individual participants, treating them as
independent entities and frequently overlooking the connections between these
participants. Unfortunately, this approach has proven less effective in
detecting important objects in complex scenarios. In response, we introduce
Driving scene Relationship self-Understanding transformer (DRUformer), designed
to enhance the important object detection task. The DRUformer is a
transformer-based multi-modal important object detection model that takes into
account the relationships between all the participants in the driving scenario.
Recognizing that driving intention also significantly affects the detection of
important objects during driving, we have incorporated a module for embedding
driving intention. To assess the performance of our approach, we conducted a
comparative experiment on the DRAMA dataset, pitting our model against other
state-of-the-art (SOTA) models. The results demonstrated a noteworthy 16.2\%
improvement in mIoU and a substantial 12.3\% boost in ACC compared to SOTA
methods. Furthermore, we conducted a qualitative analysis of our model's
ability to detect important objects across different road scenarios and
classes, highlighting its effectiveness in diverse contexts. Finally, we
conducted various ablation studies to assess the efficiency of the proposed
modules in our DRUformer model.

交通事故频繁导致致命伤害，自 2023 年以来已造成超过 5000 万人死亡。为了减轻驾驶危险并确保个人安全，有助于帮助车辆在行驶过程中预测重要物体。为此我们引入了 DRUformer 模型，它是基于 Transformer 的多模态重要物体检测模型，考虑了驾驶场景中所有参与者之间的关系，并嵌入了驾驶意图模块。我们在 DRAMA 数据集上进行了比较实验，结果显示与其他最先进模型相比，mIoU 有 16.2％的显著提高，ACC 有 12.3％的大幅提升。此外，我们还进行了定性分析，突出了我们模型在不同道路情景和类别中检测重要物体的有效性。最后，我们进行了多种消融实验，评估了 DRUformer 模型中所提出的模块的效率。

DRUformer: 提升驾驶场景中重要物体检测的驾驶关系自我理解

DRUformer: Enhancing the driving scene Important object detection with  driving relationship self-understanding

A first-person camera, placed at a person's head, captures, which objects are
important to the camera wearer. Most prior methods for this task learn to
detect such important objects from the manually labeled first-person data in a
supervised fashion. However, important objects are strongly related to the
camera wearer's internal state such as his intentions and attention, and thus,
only the person wearing the camera can provide the importance labels. Such a
constraint makes the annotation process costly and limited in scalability.
In this work, we show that we can detect important objects in first-person
images without the supervision by the camera wearer or even third-person
labelers. We formulate an important detection problem as an interplay between
the 1) segmentation and 2) recognition agents. The segmentation agent first
proposes a possible important object segmentation mask for each image, and then
feeds it to the recognition agent, which learns to predict an important object
mask using visual semantics and spatial features.
We implement such an interplay between both agents via an alternating
cross-pathway supervision scheme inside our proposed Visual-Spatial Network
(VSN). Our VSN consists of spatial ("where") and visual ("what") pathways, one
of which learns common visual semantics while the other focuses on the spatial
location cues. Our unsupervised learning is accomplished via a cross-pathway
supervision, where one pathway feeds its predictions to a segmentation agent,
which proposes a candidate important object segmentation mask that is then used
by the other pathway as a supervisory signal. We show our method's success on
two different important object datasets, where our method achieves similar or
better results as the supervised methods.

本研究提出了一种基于交替网络和路径监督的无监督学习方法，不需要依赖摄像机佩戴者或第三方注释者的重要目标检测，该方法在两个不同的数据集上实现了与有监督学习类似或更好的结果。