Monocular 3D object detection plays a crucial role in autonomous driving.
However, existing monocular 3D detection algorithms depend on 3D labels derived
from LiDAR measurements, which are costly to acquire for new datasets and
challenging to deploy in novel environments. Specifically, this study
investigates the pipeline for training a monocular 3D object detection model on
a diverse collection of 3D and 2D datasets. The proposed framework comprises
three components: (1) a robust monocular 3D model capable of functioning across
various camera settings, (2) a selective-training strategy to accommodate
datasets with differing class annotations, and (3) a pseudo 3D training
approach using 2D labels to enhance detection performance in scenes containing
only 2D labels. With this framework, we could train models on a joint set of
various open 3D/2D datasets to obtain models with significantly stronger
generalization capability and enhanced performance on new dataset with only 2D
labels. We conduct extensive experiments on
KITTI/nuScenes/ONCE/Cityscapes/BDD100K datasets to demonstrate the scaling
ability of the proposed method.

通过训练一个鲁棒的单目三维物体检测模型，采用包括多样化的三维和二维数据集、有选择地训练不同类别注释的数据集和使用二维标签的伪三维训练方法，我们可以获得具有强大泛化能力和对只有二维标签的新数据集表现出色的模型。