Fine-grained visual categorization (FGVC) aims at recognizing objects from
similar subordinate categories, which is challenging and practical for human's
accurate automatic recognition needs. Most FGVC approaches focus on the
attention mechanism research for discriminative regions mining while neglecting
their interdependencies and composed holistic object structure, which are
essential for model's discriminative information localization and understanding
ability. To address the above limitations, we propose the Structure Information
Modeling Transformer (SIM-Trans) to incorporate object structure information
into transformer for enhancing discriminative representation learning to
contain both the appearance information and structure information.
Specifically, we encode the image into a sequence of patch tokens and build a
strong vision transformer framework with two well-designed modules: (i) the
structure information learning (SIL) module is proposed to mine the spatial
context relation of significant patches within the object extent with the help
of the transformer's self-attention weights, which is further injected into the
model for importing structure information; (ii) the multi-level feature
boosting (MFB) module is introduced to exploit the complementary of multi-level
features and contrastive learning among classes to enhance feature robustness
for accurate recognition. The proposed two modules are light-weighted and can
be plugged into any transformer network and trained end-to-end easily, which
only depends on the attention weights that come with the vision transformer
itself. Extensive experiments and analyses demonstrate that the proposed
SIM-Trans achieves state-of-the-art performance on fine-grained visual
categorization benchmarks. The code is available at
this https URL

本文提出了一种结构信息建模 transformer 方法（SIM-Trans），以增强 Fine-grained visual categorization 任务中物体的判别表征学习，其中引入了两个轻量级并易于端到端训练的模块：结构信息学习模块（SIL）和多级特征增强模块（MFB），该方法在 Fei-grained 视觉分类基准测试数据集上实现了最先进的性能。