Multi-modal 3D object detection has been an active research topic in
autonomous driving. Nevertheless, it is non-trivial to explore the cross-modal
feature fusion between sparse 3D points and dense 2D pixels. Recent approaches
either fuse the image features with the point cloud features that are projected
onto the 2D image plane or combine the sparse point cloud with dense image
pixels. These fusion approaches often suffer from severe information loss, thus
causing sub-optimal performance. To address these problems, we construct the
homogeneous structure between the point cloud and images to avoid projective
information loss by transforming the camera features into the LiDAR 3D space.
In this paper, we propose a homogeneous multi-modal feature fusion and
interaction method (HMFI) for 3D object detection. Specifically, we first
design an image voxel lifter module (IVLM) to lift 2D image features into the
3D space and generate homogeneous image voxel features. Then, we fuse the
voxelized point cloud features with the image features from different regions
by introducing the self-attention based query fusion mechanism (QFM). Next, we
propose a voxel feature interaction module (VFIM) to enforce the consistency of
semantic information from identical objects in the homogeneous point cloud and
image voxel representations, which can provide object-level alignment guidance
for cross-modal feature fusion and strengthen the discriminative ability in
complex backgrounds. We conduct extensive experiments on the KITTI and Waymo
Open Dataset, and the proposed HMFI achieves better performance compared with
the state-of-the-art multi-modal methods. Particularly, for the 3D detection of
cyclist on the KITTI benchmark, HMFI surpasses all the published algorithms by
a large margin.

本研究提出了一种基于均质结构和自注意力机制的多模态特征融合和交互方法，通过将相机特征转换为 LiDAR 3D 空间，在避免项目信息丢失的同时增强了在复杂背景下的鉴别能力，在 KITTI 基准中超越了所有已发表的算法。