In this paper, we propose a monocular 3D object detection framework in the
domain of autonomous driving. Unlike previous image-based methods which focus
on RGB feature extracted from 2D images, our method solves this problem in the
reconstructed 3D space in order to exploit 3D contexts explicitly. To this end,
we first leverage a stand-alone module to transform the input data from 2D
image plane to 3D point clouds space for a better input representation, then we
perform the 3D detection using PointNet backbone net to obtain objects 3D
locations, dimensions and orientations. To enhance the discriminative
capability of point clouds, we propose a multi-modal feature fusion module to
embed the complementary RGB cue into the generated point clouds representation.
We argue that it is more effective to infer the 3D bounding boxes from the
generated 3D scene space (i.e., X,Y, Z space) compared to the image plane
(i.e., R,G,B image plane). Evaluation on the challenging KITTI dataset shows
that our approach boosts the performance of state-of-the-art monocular approach
by a large margin.

本文提出了一种自动驾驶中单目三维物体检测框架，着重解决了二维图像数据的不足，通过将输入数据从二维图像平面转换为三维点云空间进行处理，并使用 PointNet 网络进行三维检测，以提高点云的辨别能力，同时利用多模态融合模块将 RGB 颜色信息融入点云表示，证明了在三维空间中推断三维包围盒比在二维图像平面中更加有效，经过在 KITTI 数据集上的评估，该方法的表现超过现有最新的单目方法。