Interpreting machine learning model decisions is crucial for high-risk
applications like healthcare. In digital pathology, large whole slide images
(WSIs) are decomposed into smaller tiles and tile-derived features are
processed by attention-based multiple instance learning (ABMIL) models to
predict WSI-level labels. These networks generate tile-specific attention
weights, which can be visualized as attention maps for interpretability.
However, a standardized evaluation framework for these maps is lacking,
questioning their reliability and ability to detect spurious correlations that
can mislead models. We herein propose a framework to assess the ability of
attention networks to attend to relevant features in digital pathology by
creating artificial model confounders and using dedicated interpretability
metrics. Models are trained and evaluated on data with tile modifications
correlated with WSI labels, enabling the analysis of model sensitivity to
artificial confounders and the accuracy of attention maps in highlighting them.
Confounders are introduced either through synthetic tile modifications or
through tile ablations based on their specific image-based features, with the
latter being used to assess more clinically relevant scenarios. We also analyze
the impact of varying confounder quantities at both the tile and WSI levels.
Our results show that ABMIL models perform as desired within our framework.
While attention maps generally highlight relevant regions, their robustness is
affected by the type and number of confounders. Our versatile framework has the
potential to be used in the evaluation of various methods and the exploration
of image-based features driving model predictions, which could aid in biomarker
discovery.

通过引入人工模型混淆因素和使用专用的可解释性评估指标，我们提出了一个框架来评估数字病理学中注意力网络对相关特征的关注能力，发现注意力图的鲁棒性受到混淆因素的类型和数量的影响，该框架可用于评估各种方法并探索驱动模型预测的基于图像的特征，可能有助于生物标志物的发现。

数字病理学中注意力图的可解释性特征研究

Characterizing the Interpretability of Attention Maps in Digital  Pathology

Recent advances in attention-based multiple instance learning (MIL) have
improved our insights into the tissue regions that models rely on to make
predictions in digital pathology. However, the interpretability of these
approaches is still limited. In particular, they do not report whether
high-attention regions are positively or negatively associated with the class
labels or how well these regions correspond to previously established clinical
and biological knowledge. We address this by introducing a post-training
methodology to analyse MIL models. Firstly, we introduce
prediction-attention-weighted (PAW) maps by combining tile-level attention and
prediction scores produced by a refined encoder, allowing us to quantify the
predictive contribution of high-attention regions. Secondly, we introduce a
biological feature instantiation technique by integrating PAW maps with nuclei
segmentation masks. This further improves interpretability by providing
biologically meaningful features related to the cellular organisation of the
tissue and facilitates comparisons with known clinical features. We illustrate
the utility of our approach by comparing PAW maps obtained for prostate cancer
diagnosis (i.e. samples containing malignant tissue, 381/516 tissue samples)
and prognosis (i.e. samples from patients with biochemical recurrence following
surgery, 98/663 tissue samples) in a cohort of patients from the international
cancer genome consortium (ICGC UK Prostate Group). Our approach reveals that
regions that are predictive of adverse prognosis do not tend to co-locate with
the tumour regions, indicating that non-cancer cells should also be studied
when evaluating prognosis.

通过引入预测注意力加权地图和生物特征实例化技术，通过分析 MIL 模型，我们提高了对数字病理学中预测准确性区域的解释性，揭示了前列腺癌诊断中对预后有不良预测作用的区域与肿瘤区域不重合，提出了评估预后时也应研究非癌细胞。