We propose DeepFusion, a modular multi-modal architecture to fuse lidars,
cameras and radars in different combinations for 3D object detection.
Specialized feature extractors take advantage of each modality and can be
exchanged easily, making the approach simple and flexible. Extracted features
are transformed into bird's-eye-view as a common representation for fusion.
Spatial and semantic alignment is performed prior to fusing modalities in the
feature space. Finally, a detection head exploits rich multi-modal features for
improved 3D detection performance. Experimental results for lidar-camera,
lidar-camera-radar and camera-radar fusion show the flexibility and
effectiveness of our fusion approach. In the process, we study the largely
unexplored task of faraway car detection up to 225 meters, showing the benefits
of our lidar-camera fusion. Furthermore, we investigate the required density of
lidar points for 3D object detection and illustrate implications at the example
of robustness against adverse weather conditions. Moreover, ablation studies on
our camera-radar fusion highlight the importance of accurate depth estimation.

DeepFusion 提出了一种模块化的多模态架构，用于融合 lidar，相机和雷达以进行 3D 物体检测，实验结果证明了其灵活性和有效性，并探讨了远距离汽车检测和所需的激光点密度对 3D 物体检测的影响。