Lidar-based sensing drives current autonomous vehicles. Despite rapid
progress, current Lidar sensors still lag two decades behind traditional color
cameras in terms of resolution and cost. For autonomous driving, this means
that large objects close to the sensors are easily visible, but far-away or
small objects comprise only one measurement or two. This is an issue,
especially when these objects turn out to be driving hazards. On the other
hand, these same objects are clearly visible in onboard RGB sensors. In this
work, we present an approach to seamlessly fuse RGB sensors into Lidar-based 3D
recognition. Our approach takes a set of 2D detections to generate dense 3D
virtual points to augment an otherwise sparse 3D point cloud. These virtual
points naturally integrate into any standard Lidar-based 3D detectors along
with regular Lidar measurements. The resulting multi-modal detector is simple
and effective. Experimental results on the large-scale nuScenes dataset show
that our framework improves a strong CenterPoint baseline by a significant 6.6
mAP, and outperforms competing fusion approaches. Code and more visualizations
are available at this https URL

通过无缝融合 RGB 传感器到 Lidar 的 3D 识别中，我们提出了一个简单而有效的多模态检测器，该方法利用一组 2D 检测生成密集的虚拟 3D 点，能够增强原来稀疏的 3D 点云，实验结果在大规模 nuScenes 数据集上取得了显着的改进，并超过了竞争融合方法。