Autonomous driving has attracted remarkable attention from both industry and
academia. An important task is to estimate 3D properties(e.g.translation,
rotation and shape) of a moving or parked vehicle on the road. This task, while
critical, is still under-researched in the computer vision community -
partially owing to the lack of large scale and fully-annotated 3D car database
suitable for autonomous driving research. In this paper, we contribute the
first large-scale database suitable for 3D car instance understanding -
ApolloCar3D. The dataset contains 5,277 driving images and over 60K car
instances, where each car is fitted with an industry-grade 3D CAD model with
absolute model size and semantically labelled keypoints. This dataset is above
20 times larger than PASCAL3D+ and KITTI, the current state-of-the-art. To
enable efficient labelling in 3D, we build a pipeline by considering 2D-3D
keypoint correspondences for a single instance and 3D relationship among
multiple instances. Equipped with such dataset, we build various baseline
algorithms with the state-of-the-art deep convolutional neural networks.
Specifically, we first segment each car with a pre-trained Mask R-CNN, and then
regress towards its 3D pose and shape based on a deformable 3D car model with
or without using semantic keypoints. We show that using keypoints significantly
improves fitting performance. Finally, we develop a new 3D metric jointly
considering 3D pose and 3D shape, allowing for comprehensive evaluation and
ablation study. By comparing with human performance we suggest several future
directions for further improvements.

本文介绍了一种适用于自动驾驶研究的大规模 3D 汽车数据库，并提出了使用深度学习算法进行图像分割和车辆姿态与形状估计的方法，并使用关键点进一步优化模型性能。