Recently, fusing the LiDAR point cloud and camera image to improve the
performance and robustness of 3D object detection has received more and more
attention, as these two modalities naturally possess strong complementarity. In
this paper, we propose EPNet++ for multi-modal 3D object detection by
introducing a novel Cascade Bi-directional Fusion~(CB-Fusion) module and a
Multi-Modal Consistency~(MC) loss. More concretely, the proposed CB-Fusion
module enhances point features with plentiful semantic information absorbed
from the image features in a cascade bi-directional interaction fusion manner,
leading to more powerful and discriminative feature representations. The MC
loss explicitly guarantees the consistency between predicted scores from two
modalities to obtain more comprehensive and reliable confidence scores. The
experimental results on the KITTI, JRDB and SUN-RGBD datasets demonstrate the
superiority of EPNet++ over the state-of-the-art methods. Besides, we emphasize
a critical but easily overlooked problem, which is to explore the performance
and robustness of a 3D detector in a sparser scene. Extensive experiments
present that EPNet++ outperforms the existing SOTA methods with remarkable
margins in highly sparse point cloud cases, which might be an available
direction to reduce the expensive cost of LiDAR sensors. Code is available at:
this https URL

本文提出了一种名为 EPNet++ 的多模态 3D 物体检测方法，通过引入级联的双向融合 (CB-Fusion) 模块和多模态一致性 (MC) 损失，将 LiDAR 点云和相机图像进行融合，产生更强大和有区分度的特征表示，从而明显超过现有的最新技术方法。

EPNet++：级联双向融合的多模态三维物体检测

EPNet++: Cascade Bi-directional Fusion for Multi-Modal 3D Object  Detection

In this paper, we aim at addressing two critical issues in the 3D detection
task, including the exploitation of multiple sensors~(namely LiDAR point cloud
and camera image), as well as the inconsistency between the localization and
classification confidence. To this end, we propose a novel fusion module to
enhance the point features with semantic image features in a point-wise manner
without any image annotations. Besides, a consistency enforcing loss is
employed to explicitly encourage the consistency of both the localization and
classification confidence. We design an end-to-end learnable framework named
EPNet to integrate these two components. Extensive experiments on the KITTI and
SUN-RGBD datasets demonstrate the superiority of EPNet over the
state-of-the-art methods. Codes and models are available at:
https://github.com/happinesslz/EPNet.

本文提出 EPNet 框架，采用多种传感器与图像信息进行目标检测，并通过融合模块增强语义化特征，实现目标定位和分类置信度的一致性，实验表明其优于现有最先进的方法。