Object categories inherently form a hierarchy with different levels of
concept abstraction, especially for fine-grained categories. For example, birds
(Aves) can be categorized according to a four-level hierarchy of order, family,
genus, and species. This hierarchy encodes rich correlations among various
categories across different levels, which can effectively regularize the
semantic space and thus make prediction less ambiguous. However, previous
studies of fine-grained image recognition primarily focus on categories of one
certain level and usually overlook this correlation information. In this work,
we investigate simultaneously predicting categories of different levels in the
hierarchy and integrating this structured correlation information into the deep
neural network by developing a novel Hierarchical Semantic Embedding (HSE)
framework. Specifically, the HSE framework sequentially predicts the category
score vector of each level in the hierarchy, from highest to lowest. At each
level, it incorporates the predicted score vector of the higher level as prior
knowledge to learn finer-grained feature representation. During training, the
predicted score vector of the higher level is also employed to regularize label
prediction by using it as soft targets of corresponding sub-categories. To
evaluate the proposed framework, we organize the 200 bird species of the
Caltech-UCSD birds dataset with the four-level category hierarchy and construct
a large-scale butterfly dataset that also covers four level categories.
Extensive experiments on these two and the newly-released VegFru datasets
demonstrate the superiority of our HSE framework over the baseline methods and
existing competitors.

通过 Hierarchical Semantic Embedding (HSE) 框架在深层神经网络中整合不同层级的类别预测，从而实现对复杂图像的精细分类，该方法优于基线模型和现有竞争对手，可以应用于精细图像识别领域。