Robust 3D object detection is critical for safe autonomous driving. Camera and radar sensors are synergistic as they capture complementary information and work well under different environmental conditions. Fusing camera and radar data is challenging, however, as each of the sensors lacks information along a perpendicular axis, that is, depth is unknown to camera and elevation is unknown to radar. We propose the camera-radar matching network CramNet, an efficient approach to fuse the sensor readings from camera and radar in a joint 3D space. To leverage radar range measurements for better camera depth predictions, we propose a novel ray-constrained cross-attention mechanism that resolves the ambiguity in the geometric correspondences between camera features and radar features. Our method supports training with sensor modality dropout, which leads to robust 3D~object detection, even when a camera or radar sensor suddenly malfunctions on a vehicle. We demonstrate the effectiveness of our fusion approach through extensive experiments on the RADIATE dataset, one of the few large-scale datasets that provide radar radio frequency imagery. A camera-only variant of our method achieves competitive performance in monocular 3D~object detection on the Waymo Open Dataset.

本文介绍了一种利用CramNet网络实现相机和雷达传感器融合的方法，其中利用限制射线的交叉注意机制解决了相机特征和雷达特征之间几何对应关系的歧义，支持传感器模态退出训练，从而实现了鲁棒的三维物体检测。

CramNet：使用射线限制交叉注意力进行相机雷达融合，实现强大的3D物体检测