Deep learning models have achieved high performance in medical applications,
however, their adoption in clinical practice is hindered due to their black-box
nature. Self-explainable models, like prototype-based models, can be especially
beneficial as they are interpretable by design. However, if the learnt
prototypes are of low quality then the prototype-based models are as good as
black-box. Having high quality prototypes is a pre-requisite for a truly
interpretable model. In this work, we propose a prototype evaluation framework
for coherence (PEF-C) for quantitatively evaluating the quality of the
prototypes based on domain knowledge. We show the use of PEF-C in the context
of breast cancer prediction using mammography. Existing works on
prototype-based models on breast cancer prediction using mammography have
focused on improving the classification performance of prototype-based models
compared to black-box models and have evaluated prototype quality through
anecdotal evidence. We are the first to go beyond anecdotal evidence and
evaluate the quality of the mammography prototypes systematically using our
PEF-C. Specifically, we apply three state-of-the-art prototype-based models,
ProtoPNet, BRAIxProtoPNet++ and PIP-Net on mammography images for breast cancer
prediction and evaluate these models w.r.t. i) classification performance, and
ii) quality of the prototypes, on three public datasets. Our results show that
prototype-based models are competitive with black-box models in terms of
classification performance, and achieve a higher score in detecting ROIs.
However, the quality of the prototypes are not yet sufficient and can be
improved in aspects of relevance, purity and learning a variety of prototypes.
We call the XAI community to systematically evaluate the quality of the
prototypes to check their true usability in high stake decisions and improve
such models further.

通过原型评估框架 (PeF-C) 定量评估领域知识基础上的原型质量，我们首次系统评估使用我们的 PEF-C 定制的乳腺癌医学图像原型，发现原型质量有待改善以提高相关性、纯度和学习多样性。

基于原型的可解释性乳腺癌预测模型：分析与挑战

Prototype-based Interpretable Breast Cancer Prediction Models: Analysis  and Challenges

Explainable AI (XAI) has unfolded in two distinct research directions with,
on the one hand, post-hoc methods that explain the predictions of a pre-trained
black-box model and, on the other hand, self-explainable models (SEMs) which
are trained directly to provide explanations alongside their predictions. While
the latter is preferred in most safety-critical scenarios, post-hoc approaches
have received the majority of attention until now, owing to their simplicity
and ability to explain base models without retraining. Current SEMs instead,
require complex architectures and heavily regularized loss functions, thus
necessitating specific and costly training. To address this shortcoming and
facilitate wider use of SEMs, we propose a simple yet efficient universal
method called KMEx (K-Means Explainer), which can convert any existing
pre-trained model into a prototypical SEM. The motivation behind KMEx is to
push towards more transparent deep learning-based decision-making via
class-prototype-based explanations that are guaranteed to be diverse and
trustworthy without retraining the base model. We compare models obtained from
KMEx to state-of-the-art SEMs using an extensive qualitative evaluation to
highlight the strengths and weaknesses of each model, further paving the way
toward a more reliable and objective evaluation of SEMs.

通过 K-Means Explainer（KMEx）方法，将任何现有的预训练模型转换为原型式可自解释模型，以实现更透明的基于深度学习的决策，通过无需重新训练基础模型的类原型解释，保证解释的多样性和可信度，提供了一种简单但高效的普适方法。