Vision Transformers are at the heart of the current surge of interest in
foundation models for histopathology. They process images by breaking them into
smaller patches following a regular grid, regardless of their content. Yet, not
all parts of an image are equally relevant for its understanding. This is
particularly true in computational pathology where background is completely
non-informative and may introduce artefacts that could mislead predictions. To
address this issue, we propose a novel method that explicitly masks background
in Vision Transformers' attention mechanism. This ensures tokens corresponding
to background patches do not contribute to the final image representation,
thereby improving model robustness and interpretability. We validate our
approach using prostate cancer grading from whole-slide images as a case study.
Our results demonstrate that it achieves comparable performance with plain
self-attention while providing more accurate and clinically meaningful
attention heatmaps.

通过在注意机制中使用背景遮罩的新方法，视觉 Transformers 在计算病理学中的背景抑制中取得了可比的性能，提供了更准确和临床意义的注意热图。