Many recent works on 3D object detection have focused on designing neural
network architectures that can consume point cloud data. While these approaches
demonstrate encouraging performance, they are typically based on a single
modality and are unable to leverage information from other modalities, such as
a camera. Although a few approaches fuse data from different modalities, these
methods either use a complicated pipeline to process the modalities
sequentially, or perform late-fusion and are unable to learn interaction
between different modalities at early stages. In this work, we present
PointFusion and VoxelFusion: two simple yet effective early-fusion approaches
to combine the RGB and point cloud modalities, by leveraging the recently
introduced VoxelNet architecture. Evaluation on the KITTI dataset demonstrates
significant improvements in performance over approaches which only use point
cloud data. Furthermore, the proposed method provides results competitive with
the state-of-the-art multimodal algorithms, achieving top-2 ranking in five of
the six bird's eye view and 3D detection categories on the KITTI benchmark, by
using a simple single stage network.

该研究提出了两种简单而有效的融合 RGB 和点云模态的早期融合方法，称为 PointFusion 和 VoxelFusion，并利用 VoxelNet 结构来结合这些模态，从而实现与最先进的多模态算法竞争力相当的表现，其能够在 KITTI 基准测试的五个车顶俯视图和 3D 检测类别中实现前两名的排名。