Recently, Transformers have shown promising performance in various vision
tasks. However, the high costs of global self-attention remain challenging for
Transformers, especially for high-resolution vision tasks. Local self-attention
runs attention computation within a limited region for the sake of efficiency,
resulting in insufficient context modeling as their receptive fields are small.
In this work, we introduce two new attention modules to enhance the global
modeling capability of the hierarchical vision transformer, namely, random
sampling windows (RS-Win) and important region windows (IR-Win). Specifically,
RS-Win sample random image patches to compose the window, following a uniform
distribution, i.e., the patches in RS-Win can come from any position in the
image. IR-Win composes the window according to the weights of the image patches
in the attention map. Notably, RS-Win is able to capture global information
throughout the entire model, even in earlier, high-resolution stages. IR-Win
enables the self-attention module to focus on important regions of the image
and capture more informative features. Incorporated with these designs,
RSIR-Win Transformer demonstrates competitive performance on common vision
tasks.

本文介绍了 RS-Win 和 IR-Win 两种新的注意力机制，用于增强分层视觉 Transformer 的全局建模能力，同时也提高了自我注意模块对图像重要区域的关注能力，这个新的设计有助于在通用视觉任务中提高 RSIR-Win Transformer 的性能。