Vision-based autonomous driving requires reliable and efficient object
detection. This work proposes a DiffusionDet-based framework that exploits data
fusion from the monocular camera and depth sensor to provide the RGB and depth
(RGB-D) data. Within this framework, ground truth bounding boxes are randomly
reshaped as part of the training phase, allowing the model to learn the reverse
diffusion process of noise addition. The system methodically enhances a
randomly generated set of boxes at the inference stage, guiding them toward
accurate final detections. By integrating the textural and color features from
RGB images with the spatial depth information from the LiDAR sensors, the
proposed framework employs a feature fusion that substantially enhances object
detection of automotive targets. The $2.3$ AP gain in detecting automotive
targets is achieved through comprehensive experiments using the KITTI dataset.
Specifically, the improved performance of the proposed approach in detecting
small objects is demonstrated.

一项基于视觉的自动驾驶需要可靠高效的对象检测的研究提出了一种 DiffusionDet 框架，该框架利用单目相机和深度传感器的数据融合来提供 RGB 和深度（RGB-D）数据，并通过在训练阶段随机重塑基准边界框，使模型学习噪声加入的相反扩散过程。通过将 RGB 图像的纹理和颜色特征与 LiDAR 传感器的空间深度信息结合起来，所提出的框架采用了特征融合，从而大大提高了汽车目标的对象检测能力。在 KITTI 数据集上进行的全面实验取得了 2.3 的 AP 增益，特别是在检测小物体方面展示了所提出方法的改进性能。

使用 RGB-D 融合在 DiffusionDet 框架中增强的汽车物体检测

Enhanced Automotive Object Detection via RGB-D Fusion in a DiffusionDet  Framework

Vision-based urban driving is hard. The autonomous system needs to learn to
perceive the world and act in it. We show that this challenging learning
problem can be simplified by decomposing it into two stages. We first train an
agent that has access to privileged information. This privileged agent cheats
by observing the ground-truth layout of the environment and the positions of
all traffic participants. In the second stage, the privileged agent acts as a
teacher that trains a purely vision-based sensorimotor agent. The resulting
sensorimotor agent does not have access to any privileged information and does
not cheat. This two-stage training procedure is counter-intuitive at first, but
has a number of important advantages that we analyze and empirically
demonstrate. We use the presented approach to train a vision-based autonomous
driving system that substantially outperforms the state of the art on the CARLA
benchmark and the recent NoCrash benchmark. Our approach achieves, for the
first time, 100% success rate on all tasks in the original CARLA benchmark,
sets a new record on the NoCrash benchmark, and reduces the frequency of
infractions by an order of magnitude compared to the prior state of the art.
For the video that summarizes this work, see this https URL

本文提出了一种基于视觉的自主行驶系统的训练方法，通过两个阶段的训练，先使用有特权的信息训练代理，再使用纯视觉信息训练传感器驱动代理，最终取得了在 CARLA 基准和 NoCrash 基准测试中的优异成绩。