Current multi-modal object detection approaches focus on the vehicle domain
and are limited in the perception range and the processing capabilities.
Roadside sensor units (RSUs) introduce a new domain for perception systems and
leverage altitude to observe traffic. Cameras and LiDARs mounted on gantry
bridges increase the perception range and produce a full digital twin of the
traffic. In this work, we introduce InfraDet3D, a multi-modal 3D object
detector for roadside infrastructure sensors. We fuse two LiDARs using early
fusion and further incorporate detections from monocular cameras to increase
the robustness and to detect small objects. Our monocular 3D detection module
uses HD maps to ground object yaw hypotheses, improving the final perception
results. The perception framework is deployed on a real-world intersection that
is part of the A9 Test Stretch in Munich, Germany. We perform several ablation
studies and experiments and show that fusing two LiDARs with two cameras leads
to an improvement of +1.90 mAP compared to a camera-only solution. We evaluate
our results on the A9 infrastructure dataset and achieve 68.48 mAP on the test
set. The dataset and code will be available at this https URL to allow
the research community to further improve the perception results and make
autonomous driving safer.

本文介绍了一种新的基础设施传感器的多模 3D 物体检测器 InfraDet3D，有效结合了两个 LiDAR 和单目相机数据，通过早期融合方法提高检测效果，并将 HD 地图应用于单目 3D 检测模块，以提升其最终的感知结果；该系统在德国慕尼黑的 A9 测试区域进行了实际部署和评估，得到了 68.48 的 mAP 测试结果，同时数据集和代码也提供给研究社区进行进一步的研究。

InfraDet3D：基于路侧基础设施摄像机和激光雷达传感器的多模态三维物体检测

InfraDet3D: Multi-Modal 3D Object Detection based on Roadside  Infrastructure Camera and LiDAR Sensors

As the perception range of LiDAR increases, LiDAR-based 3D object detection
becomes a dominant task in the long-range perception task of autonomous
driving. The mainstream 3D object detectors usually build dense feature maps in
the network backbone and prediction head. However, the computational and
spatial costs on the dense feature map are quadratic to the perception range,
which makes them hardly scale up to the long-range setting. To enable efficient
long-range LiDAR-based object detection, we build a fully sparse 3D object
detector (FSD). The computational and spatial cost of FSD is roughly linear to
the number of points and independent of the perception range. FSD is built upon
the general sparse voxel encoder and a novel sparse instance recognition (SIR)
module. SIR first groups the points into instances and then applies
instance-wise feature extraction and prediction. In this way, SIR resolves the
issue of center feature missing, which hinders the design of the fully sparse
architecture for all center-based or anchor-based detectors. Moreover, SIR
avoids the time-consuming neighbor queries in previous point-based methods by
grouping points into instances. We conduct extensive experiments on the
large-scale Waymo Open Dataset to reveal the working mechanism of FSD, and
state-of-the-art performance is reported. To demonstrate the superiority of FSD
in long-range detection, we also conduct experiments on Argoverse 2 Dataset,
which has a much larger perception range ($200m$) than Waymo Open Dataset
($75m$). On such a large perception range, FSD achieves state-of-the-art
performance and is 2.4$\times$ faster than the dense counterpart. Codes will be
released at this https URL

通过稀疏编码器与稀疏实例识别（SIR）模块，提出了一种全稀疏三维目标检测器（FSD），该方法可以在自动驾驶的远距离感知任务中实现高效的检测，具有状态 - of-the-art 性能并且比密集对应物快 2.4 倍。