Despite the impressive representation capacity of vision transformer models,
current light-weight vision transformer models still suffer from inconsistent
and incorrect dense predictions at local regions. We suspect that the power of
their self-attention mechanism is limited in shallower and thinner networks. We
propose Lite Vision Transformer (LVT), a novel light-weight transformer network
with two enhanced self-attention mechanisms to improve the model performances
for mobile deployment. For the low-level features, we introduce Convolutional
Self-Attention (CSA). Unlike previous approaches of merging convolution and
self-attention, CSA introduces local self-attention into the convolution within
a kernel of size 3x3 to enrich low-level features in the first stage of LVT.
For the high-level features, we propose Recursive Atrous Self-Attention (RASA),
which utilizes the multi-scale context when calculating the similarity map and
a recursive mechanism to increase the representation capability with marginal
extra parameter cost. The superiority of LVT is demonstrated on ImageNet
recognition, ADE20K semantic segmentation, and COCO panoptic segmentation. The
code is made publicly available.

本研究提出了 Lite Vision Transformer（LVT），一种新型的轻量级 transformer 网络，其中包含两种增强自注意力机制，以提高模型在移动部署中的性能表现，并在 ImageNet 识别、ADE20K 语义分割和 COCO 全景分割中具有优越性能。