We present a novel learning framework for vehicle recognition from a single
RGB image. Unlike existing methods which only use attention mechanisms to
locate 2D discriminative information, our work learns a novel 3D perspective
feature representation of a vehicle, which is then fused with 2D appearance
feature to predict the category. The framework is composed of a global network
(GN), a 3D perspective network (3DPN), and a fusion network. The GN is used to
locate the region of interest (RoI) and generate the 2D global feature. With
the assistance of the RoI, the 3DPN estimates the 3D bounding box under the
guidance of the proposed vanishing point loss, which provides a perspective
geometry constraint. Then the proposed 3D representation is generated by
eliminating the viewpoint variance of the 3D bounding box using perspective
transformation. Finally, the 3D and 2D feature are fused to predict the
category of the vehicle. We present qualitative and quantitative results on the
vehicle classification and verification tasks in the BoxCars dataset. The
results demonstrate that, by learning such a concise 3D representation, we can
achieve superior performance to methods that only use 2D information while
retain 3D meaningful information without the challenge of requiring a 3D CAD
model.

本研究基于车辆的单一 RGB 图像，提出了一种学习框架，其将 3D 视角的特征表示与 2D 外观特征相结合进行车辆识别，通过在 BoxCars 数据集上的分类和验证任务中取得优越性能的结果验证了该方法的有效性。