Current LiDAR-only 3D detection methods inevitably suffer from the sparsity
of point clouds. Many multi-modal methods are proposed to alleviate this issue,
while different representations of images and point clouds make it difficult to
fuse them, resulting in suboptimal performance. In this paper, we present a
novel multi-modal framework SFD (Sparse Fuse Dense), which utilizes pseudo
point clouds generated from depth completion to tackle the issues mentioned
above. Different from prior works, we propose a new RoI fusion strategy 3D-GAF
(3D Grid-wise Attentive Fusion) to make fuller use of information from
different types of point clouds. Specifically, 3D-GAF fuses 3D RoI features
from the couple of point clouds in a grid-wise attentive way, which is more
fine-grained and more precise. In addition, we propose a SynAugment
(Synchronized Augmentation) to enable our multi-modal framework to utilize all
data augmentation approaches tailored to LiDAR-only methods. Lastly, we
customize an effective and efficient feature extractor CPConv (Color Point
Convolution) for pseudo point clouds. It can explore 2D image features and 3D
geometric features of pseudo point clouds simultaneously. Our method holds the
highest entry on the KITTI car 3D object detection leaderboard, demonstrating
the effectiveness of our SFD. Codes are available at
this https URL

本文提出了一种基于 SFD 框架的多模态方法，采用由深度补全生成的伪点云来解决点云稀疏性问题，并提出一种新的 3D RoI 融合策略 3D-GAF 来更好地利用不同类型点云的信息。同时，本文还介绍了通过数据增强等措施让多模态框架利用 LIDAR-only 方法。最后，实验表明本文所提出的方法在 KITTI 3D 目标检测排行榜上获得了最高的入口，证明了其有效性。