3D object detection based on LiDAR-camera fusion is becoming an emerging
research theme for autonomous driving. However, it has been surprisingly
difficult to effectively fuse both modalities without information loss and
interference. To solve this issue, we propose a single-stage multi-view fusion
framework that takes LiDAR bird's-eye view, LiDAR range view and camera view
images as inputs for 3D object detection. To effectively fuse multi-view
features, we propose an attentive pointwise fusion (APF) module to estimate the
importance of the three sources with attention mechanisms that can achieve
adaptive fusion of multi-view features in a pointwise manner. Furthermore, an
attentive pointwise weighting (APW) module is designed to help the network
learn structure information and point feature importance with two extra tasks,
namely, foreground classification and center regression, and the predicted
foreground probability is used to reweight the point features. We design an
end-to-end learnable network named MVAF-Net to integrate these two components.
Our evaluations conducted on the KITTI 3D object detection datasets demonstrate
that the proposed APF and APW modules offer significant performance gains.
Moreover, the proposed MVAF-Net achieves the best performance among all
single-stage fusion methods and outperforms most two-stage fusion methods,
achieving the best trade-off between speed and accuracy on the KITTI benchmark.

本文提出了一种基于多视角融合的 3D 目标检测网络，结合了激光雷达鸟瞰视角、激光雷达距离视角和相机视角图像，使用注意力机制和额外任务优化点特征和结构信息，有效地实现了多视角特征的适应性融合，该网络在 KITTI 3D 目标检测数据集上的表现令人满意。