Although explainability is essential in the clinical diagnosis, most deep
learning models still function as black boxes without elucidating their
decision-making process. In this study, we investigate the explainable model
development that can mimic the decision-making process of human experts by
fusing the domain knowledge of explicit diagnostic criteria. We introduce a
simple yet effective framework, Explicd, towards Explainable language-informed
criteria-based diagnosis. Explicd initiates its process by querying domain
knowledge from either large language models (LLMs) or human experts to
establish diagnostic criteria across various concept axes (e.g., color, shape,
texture, or specific patterns of diseases). By leveraging a pretrained
vision-language model, Explicd injects these criteria into the embedding space
as knowledge anchors, thereby facilitating the learning of corresponding visual
concepts within medical images. The final diagnostic outcome is determined
based on the similarity scores between the encoded visual concepts and the
textual criteria embeddings. Through extensive evaluation of five medical image
classification benchmarks, Explicd has demonstrated its inherent explainability
and extends to improve classification performance compared to traditional
black-box models.

通过融合领域知识与大型语言模型，Explicd 提出了一种可解释的、以诊断标准为基础的语言辅助诊断框架，不仅模拟了人类专家的决策过程，还提高了医学图像分类性能。