Recent camera-based 3D object detection is limited by the precision of
transforming from image to 3D feature spaces, as well as the accuracy of object
localization within the 3D space. This paper aims to address such a fundamental
problem of camera-based 3D object detection: How to effectively learn depth
information for accurate feature lifting and object localization. Different
from previous methods which directly predict depth distributions by using a
supervised estimation model, we propose a cascade framework consisting of two
depth-aware learning paradigms. First, a depth estimation (DE) scheme leverages
relative depth information to realize the effective feature lifting from 2D to
3D spaces. Furthermore, a depth calibration (DC) scheme introduces depth
reconstruction to further adjust the 3D object localization perturbation along
the depth axis. In practice, the DE is explicitly realized by using both the
absolute and relative depth optimization loss to promote the precision of depth
prediction, while the capability of DC is implicitly embedded into the
detection Transformer through a depth denoising mechanism in the training
phase. The entire model training is accomplished through an end-to-end manner.
We propose a baseline detector and evaluate the effectiveness of our proposal
with +2.2%/+2.7% NDS/mAP improvements on NuScenes benchmark, and gain a
comparable performance with 55.9%/45.7% NDS/mAP. Furthermore, we conduct
extensive experiments to demonstrate its generality based on various detectors
with about +2% NDS improvements.

本文提出了一个基于相机的 3D 目标检测的级联框架，通过深度估计和深度校准来有效地学习深度信息，实现从 2D 到 3D 空间的特征提取和目标定位。在多个检测器上进行广泛实验，取得了较高的性能表现。