Predicting high-dimensional or extreme multilabels, such as in medical coding, requires both accuracy and interpretability. Existing works often rely on local interpretability methods, failing to provide comprehensive explanations of the overall mechanism behind each label prediction within a multilabel set. We propose a mechanistic interpretability module called DIctionary Label Attention (\method) that disentangles uninterpretable dense embeddings into a sparse embedding space, where each nonzero element (a dictionary feature) represents a globally learned medical concept. Through human evaluations, we show that our sparse embeddings are more human understandable than its dense counterparts by at least 50 percent. Our automated dictionary feature identification pipeline, leveraging large language models (LLMs), uncovers thousands of learned medical concepts by examining and summarizing the highest activating tokens for each dictionary feature. We represent the relationships between dictionary features and medical codes through a sparse interpretable matrix, enhancing the mechanistic and global understanding of the model's predictions while maintaining competitive performance and scalability without extensive human annotation.

本研究针对多标签医疗编码预测中存在的可解释性不足问题，提出了一种称为字典标签注意力的机制可解释性模块，以解构不可解释的稠密嵌入为稀疏嵌入空间，并通过人类评估证明其可理解性显著提高。利用大语言模型的自动化字典特征识别流程，我们揭示了数千个学习到的医疗概念，增强了模型预测的机制和全球理解，同时不需广泛的人类标注，保持了竞争性能和可扩展性。

高维多标签医疗编码预测中的机制可解释性的字典标签注意力