In this work, we propose a novel single-shot and keypoints-based framework
for monocular 3D objects detection using only RGB images, called KM3D-Net. We
design a fully convolutional model to predict object keypoints, dimension, and
orientation, and then combine these estimations with perspective geometry
constraints to compute position attribute. Further, we reformulate the
geometric constraints as a differentiable version and embed it into the network
to reduce running time while maintaining the consistency of model outputs in an
end-to-end fashion. Benefiting from this simple structure, we then propose an
effective semi-supervised training strategy for the setting where labeled
training data is scarce. In this strategy, we enforce a consensus prediction of
two shared-weights KM3D-Net for the same unlabeled image under different input
augmentation conditions and network regularization. In particular, we unify the
coordinate-dependent augmentations as the affine transformation for the
differential recovering position of objects and propose a keypoints-dropout
module for the network regularization. Our model only requires RGB images
without synthetic data, instance segmentation, CAD model, or depth generator.
Nevertheless, extensive experiments on the popular KITTI 3D detection dataset
indicate that the KM3D-Net surpasses all previous state-of-the-art methods in
both efficiency and accuracy by a large margin. And also, to the best of our
knowledge, this is the first time that semi-supervised learning is applied in
monocular 3D objects detection. We even surpass most of the previous fully
supervised methods with only 13\% labeled data on KITTI.

KM3D-Net 提出了一种基于 RGB 图像和关键点的单次检测框架，通过结合完全卷积模型、几何约束和不依赖 CAD 模型、深度生成器，以及一种有效的半监督训练策略，实现了卓越的检测表现，并超过以前所有的最先进方法。