Multi-sensor modal fusion has demonstrated strong advantages in 3D object
detection tasks. However, existing methods that fuse multi-modal features
through a simple channel concatenation require transformation features into
bird's eye view space and may lose the information on Z-axis thus leads to
inferior performance. To this end, we propose FusionFormer, an end-to-end
multi-modal fusion framework that leverages transformers to fuse multi-modal
features and obtain fused BEV features. And based on the flexible adaptability
of FusionFormer to the input modality representation, we propose a depth
prediction branch that can be added to the framework to improve detection
performance in camera-based detection tasks. In addition, we propose a
plug-and-play temporal fusion module based on transformers that can fuse
historical frame BEV features for more stable and reliable detection results.
We evaluate our method on the nuScenes dataset and achieve 72.6% mAP and 75.1%
NDS for 3D object detection tasks, outperforming state-of-the-art methods.

多传感器模态融合的 FusionFormer 框架通过 transformers 混合多模态特征，获取鸟瞰图特征，并通过可灵活适应输入模态表示的深度预测分支提高相机检测任务的性能，同时提出一种基于 transformers 的即插即用的时域融合模块用于融合历史帧鸟瞰图特征，实现更稳定可靠的检测结果，通过对 nuScenes 数据集的评估，其在 3D 目标检测任务中获得了 72.6％的 mAP 和 75.1％的 NDS，优于现有的最先进方法。