We present a new way to detect 3D objects from multimodal inputs, leveraging both LiDAR and RGB cameras in a hybrid late-cascade scheme, that combines an RGB detection network and a 3D LiDAR detector. We exploit late fusion principles to reduce LiDAR False Positives, matching LiDAR detections with RGB ones by projecting the LiDAR bounding boxes on the image. We rely on cascade fusion principles to recover LiDAR False Negatives leveraging epipolar constraints and frustums generated by RGB detections of separate views. Our solution can be plugged on top of any underlying single-modal detectors, enabling a flexible training process that can take advantage of pre-trained LiDAR and RGB detectors, or train the two branches separately. We evaluate our results on the KITTI object detection benchmark, showing significant performance improvements, especially for the detection of Pedestrians and Cyclists.

本研究解决了多模态输入下3D物体检测中的假阳性和假阴性问题，提出了一种混合晚级联方案，结合了RGB检测网络和3D LiDAR检测器。通过将LiDAR边界框投影到图像上并利用RGB检测的视图生成的锥体，我们显著提高了在KITTI数据集上的检测性能，特别是对行人和骑自行车者的检测。 

增强3D物体检测的多模态混合晚级联融合网络