Black-box deep learning approaches have showcased significant potential in
the realm of medical image analysis. However, the stringent trustworthiness
requirements intrinsic to the medical field have catalyzed research into the
utilization of Explainable Artificial Intelligence (XAI), with a particular
focus on concept-based methods. Existing concept-based methods predominantly
apply concept annotations from a single perspective (e.g., global level),
neglecting the nuanced semantic relationships between sub-regions and concepts
embedded within medical images. This leads to underutilization of the valuable
medical information and may cause models to fall short in harmoniously
balancing interpretability and performance when employing inherently
interpretable architectures such as Concept Bottlenecks. To mitigate these
shortcomings, we propose a multi-modal explainable disease diagnosis framework
that meticulously aligns medical images and clinical-related concepts
semantically at multiple strata, encompassing the image level, token level, and
concept level. Moreover, our method allows for model intervention and offers
both textual and visual explanations in terms of human-interpretable concepts.
Experimental results on three skin image datasets demonstrate that our method,
while preserving model interpretability, attains high performance and label
efficiency for concept detection and disease diagnosis.

通过使用黑盒深度学习方法，并结合可解释的人工智能和基于概念的方法，在医学图像分析领域提出一种多模态可解释疾病诊断框架，以实现模型可解释性、高性能和标签效率的平衡。