Biomedical image analysis can be facilitated by an innovative architecture
rooted in self-attention mechanisms. The traditional convolutional neural
network (CNN), characterized by fixed-sized windows, needs help capturing
intricate spatial and temporal relations at the pixel level. The immutability
of CNN filter weights post-training further restricts input fluctuations.
Recognizing these limitations, we propose a new paradigm of attention-based
models instead of convolutions. As an alternative to traditional CNNs, these
models demonstrate robust modelling capabilities and the ability to grasp
comprehensive long-range contextual information efficiently. Providing a
solution to critical challenges faced by attention-based vision models such as
inductive bias, weight sharing, receptive field limitations, and data handling
in high resolution, our work combines non-overlapping (vanilla patching) with
novel overlapped Shifted Patching Techniques (S.P.T.s) to induce local context
that enhances model generalization. Moreover, we examine the novel Lancoz5
interpolation technique, which adapts variable image sizes to higher
resolutions. Experimental evidence validates our model's generalization
effectiveness, comparing favourably with existing approaches. Attention-based
methods are particularly effective with ample data, especially when advanced
data augmentation methodologies are integrated to strengthen their robustness.

医学图像分析可以通过一种扎根于自注意机制的创新架构得到促进。传统的卷积神经网络（CNN）在捕捉像素级复杂空间和时间关系时需要帮助，而其卷积窗口的固定大小限制了输入波动。为了克服这些限制，我们提出了基于注意力模型而非卷积的新范式。这些模型作为传统 CNN 的替代品，表现出强大的建模能力和高效获取全局长范围上下文信息的能力。通过将非重叠（vanilla patching）和新颖的重叠移位补丁技术（S.P.T.s）相结合，我们的工作提供了一种解决注意力视觉模型所面临的关键挑战的方法，包括归纳偏差、权重共享、感受野限制和高分辨率数据处理。此外，我们还研究了适应不同图像尺寸到更高分辨率的新颖 Lancoz5 插值技术。实验证据验证了我们模型的一般化效果，并与现有方法进行了有利比较。特别是在充足数据的情况下，基于注意力的方法特别有效，尤其是在集成先进的数据增强方法来增强它们的鲁棒性时。