We propose a novel framework for fine-grained object recognition that learns
to recover object variation in 3D space from a single image, trained on an
image collection without using any ground-truth 3D annotation. We accomplish
this by representing an object as a composition of 3D shape and its appearance,
while eliminating the effect of camera viewpoint, in a canonical configuration.
Unlike conventional methods modeling spatial variation in 2D images only, our
method is capable of reconfiguring the appearance feature in a canonical 3D
space, thus enabling the subsequent object classifier to be invariant under 3D
geometric variation. Our representation also allows us to go beyond existing
methods, by incorporating 3D shape variation as an additional cue for object
recognition. To learn the model without ground-truth 3D annotation, we deploy a
differentiable renderer in an analysis-by-synthesis framework. By incorporating
3D shape and appearance jointly in a deep representation, our method learns the
discriminative representation of the object and achieves competitive
performance on fine-grained image recognition and vehicle re-identification. We
also demonstrate that the performance of 3D shape reconstruction is improved by
learning fine-grained shape deformation in a boosting manner.

该研究提出了一种新型的精细物体识别框架，能够从一个单一的图像中学习恢复物体在三维空间中的变化，并在无需使用任何基准 3D 注释情况下，在图像集合上进行训练。