We present a novel end-to-end framework named as GSNet (Geometric and
Scene-aware Network), which jointly estimates 6DoF poses and reconstructs
detailed 3D car shapes from single urban street view. GSNet utilizes a unique
four-way feature extraction and fusion scheme and directly regresses 6DoF poses
and shapes in a single forward pass. Extensive experiments show that our
diverse feature extraction and fusion scheme can greatly improve model
performance. Based on a divide-and-conquer 3D shape representation strategy,
GSNet reconstructs 3D vehicle shape with great detail (1352 vertices and 2700
faces). This dense mesh representation further leads us to consider geometrical
consistency and scene context, and inspires a new multi-objective loss function
to regularize network training, which in turn improves the accuracy of 6D pose
estimation and validates the merit of jointly performing both tasks. We
evaluate GSNet on the largest multi-task ApolloCar3D benchmark and achieve
state-of-the-art performance both quantitatively and qualitatively. Project
page is available at this https URL

GSNet 采用独特的四向特征提取和融合方案，直接回归 6DoF 姿态和形状，实现从单眼图像中重建具有细节的三维车形。