Vision transformers (ViTs) that model an image as a sequence of partitioned
patches have shown notable performance in diverse vision tasks. Because
partitioning patches eliminates the image structure, to reflect the order of
patches, ViTs utilize an explicit component called positional embedding.
However, we claim that the use of positional embedding does not simply
guarantee the order-awareness of ViT. To support this claim, we analyze the
actual behavior of ViTs using an effective receptive field. We demonstrate that
during training, ViT acquires an understanding of patch order from the
positional embedding that is trained to be a specific pattern. Based on this
observation, we propose explicitly adding a Gaussian attention bias that guides
the positional embedding to have the corresponding pattern from the beginning
of training. We evaluated the influence of Gaussian attention bias on the
performance of ViTs in several image classification, object detection, and
semantic segmentation experiments. The results showed that proposed method not
only facilitates ViTs to understand images but also boosts their performance on
various datasets, including ImageNet, COCO 2017, and ADE20K.

本研究提出了一种使用高斯注意偏差来指导位置嵌入的方法，以提高视觉转换器（ViTs）在图像分类、目标检测和语义分割等任务中的性能。研究表明，使用这种方法不仅有助于 ViT 理解图像，还提高了在各种数据集上的性能。