We propose a 3D object detection system with multi-sensor refinement in the
context of autonomous driving. In our framework, the monocular camera serves as
the fundamental sensor for 2D object proposal and initial 3D bounding box
prediction. While the stereo cameras and LiDAR are treated as adaptive plug-in
sensors to refine the 3D box localization performance. For each observed
element in the raw measurement domain (e.g., pixels for stereo, 3D points for
LiDAR), we model the local geometry as an instance vector representation, which
indicates the 3D coordinate of each element respecting to the object frame.
Using this unified geometric representation, the 3D object location can be
unified refined by the stereo photometric alignment or point cloud alignment.
We demonstrate superior 3D detection and localization performance compared to
state-of-the-art monocular, stereo methods and competitive performance compared
with the baseline LiDAR method on the KITTI object benchmark.

本文提出了一种在自动驾驶背景下具有多传感器优化的三维目标检测系统，其中单眼相机可以作为二维目标提议和初步三维边界框预测，而立体相机和 LiDAR 则被视为自适应插件传感器以优化 3D 边界框本地化性能。