Multi-sensor fusion is essential for an accurate and reliable autonomous
driving system. Recent approaches are based on point-level fusion: augmenting
the LiDAR point cloud with camera features. However, the camera-to-LiDAR
projection throws away the semantic density of camera features, hindering the
effectiveness of such methods, especially for semantic-oriented tasks (such as
3D scene segmentation). In this paper, we break this deeply-rooted convention
with BEVFusion, an efficient and generic multi-task multi-sensor fusion
framework. It unifies multi-modal features in the shared bird's-eye view (BEV)
representation space, which nicely preserves both geometric and semantic
information. To achieve this, we diagnose and lift key efficiency bottlenecks
in the view transformation with optimized BEV pooling, reducing latency by more
than 40x. BEVFusion is fundamentally task-agnostic and seamlessly supports
different 3D perception tasks with almost no architectural changes. It
establishes the new state of the art on nuScenes, achieving 1.3% higher mAP and
NDS on 3D object detection and 13.6% higher mIoU on BEV map segmentation, with
1.9x lower computation cost. Code to reproduce our results is available at
this https URL

本研究提出 BEVFusion，这是一种有效和通用的多任务多传感器融合框架，通过在共享的鸟瞰图表示空间中统一多模态特征，从根本上支持不同的 3D 感知任务，并在 nuScenes 上创立了新的技术水平