Combining complementary sensor modalities is crucial to providing robust
perception for safety-critical robotics applications such as autonomous driving
(AD). Recent state-of-the-art camera-lidar fusion methods for AD rely on
monocular depth estimation which is a notoriously difficult task compared to
using depth information from the lidar directly. Here, we find that this
approach does not leverage depth as expected and show that naively improving
depth estimation does not lead to improvements in object detection performance
and that, strikingly, removing depth estimation altogether does not degrade
object detection performance. This suggests that relying on monocular depth
could be an unnecessary architectural bottleneck during camera-lidar fusion. In
this work, we introduce a novel fusion method that bypasses monocular depth
estimation altogether and instead selects and fuses camera and lidar features
in a bird's-eye-view grid using a simple attention mechanism. We show that our
model can modulate its use of camera features based on the availability of
lidar features and that it yields better 3D object detection on the nuScenes
dataset than baselines relying on monocular depth estimation.

通过在鸟瞰图格栅中使用简单的注意力机制选择和融合相机和激光雷达特征，我们引入了一种绕过单眼深度估计的新型融合方法，展示了在 nuScenes 数据集上我们的模型可以根据激光雷达特征的可用性调节其使用相机特征，从而在 3D 目标检测方面优于依赖单眼深度估计的基线方法。

飞升 - 参与 - 弹溅：使用 Transformer 进行鸟瞰摄像头 - 激光雷达融合

Lift-Attend-Splat: Bird's-eye-view camera-lidar fusion using  transformers

Bird's-eye-view (BEV) grid is a typical representation of the perception of
road components, e.g., drivable area, in autonomous driving. Most existing
approaches rely on cameras only to perform segmentation in BEV space, which is
fundamentally constrained by the absence of reliable depth information. The
latest works leverage both camera and LiDAR modalities but suboptimally fuse
their features using simple, concatenation-based mechanisms. In this paper, we
address these problems by enhancing the alignment of the unimodal features in
order to aid feature fusion, as well as enhancing the alignment between the
cameras' perspective view (PV) and BEV representations. We propose X-Align, a
novel end-to-end cross-modal and cross-view learning framework for BEV
segmentation consisting of the following components: (i) a novel Cross-Modal
Feature Alignment (X-FA) loss, (ii) an attention-based Cross-Modal Feature
Fusion (X-FF) module to align multi-modal BEV features implicitly, and (iii) an
auxiliary PV segmentation branch with Cross-View Segmentation Alignment (X-SA)
losses to improve the PV-to-BEV transformation. We evaluate our proposed method
across two commonly used benchmark datasets, i.e., nuScenes and KITTI-360.
Notably, X-Align significantly outperforms the state-of-the-art by 3 absolute
mIoU points on nuScenes. We also provide extensive ablation studies to
demonstrate the effectiveness of the individual components.

本文提出了一种新的跨模态和跨视角学习框架，使用基于注意力的跨模态特征融合模块和辅助 PV 分割分支，旨在提高 BEV 分割的性能。在 nuScenes 和 KITTI-360 数据集上的评估结果表明，该方法明显优于现有技术并能够更好地应用于自动驾驶。