Despite their success, Large-Language Models (LLMs) still face criticism as
their lack of interpretability limits their controllability and reliability.
Traditional post-hoc interpretation methods, based on attention and
gradient-based analysis, offer limited insight into the model's decision-making
processes. In the image field, Concept-based models have emerged as
explainable-by-design architectures, employing human-interpretable features as
intermediate representations. However, these methods have not been yet adapted
to textual data, mainly because they require expensive concept annotations,
which are impractical for real-world text data. This paper addresses this
challenge by proposing a self-supervised Interpretable Concept Embedding Models
(ICEMs). We leverage the generalization abilities of LLMs to predict the
concepts labels in a self-supervised way, while we deliver the final
predictions with an interpretable function. The results of our experiments show
that ICEMs can be trained in a self-supervised way achieving similar
performance to fully supervised concept-based models and end-to-end black-box
ones. Additionally, we show that our models are (i) interpretable, offering
meaningful logical explanations for their predictions; (ii) interactable,
allowing humans to modify intermediate predictions through concept
interventions; and (iii) controllable, guiding the LLMs' decoding process to
follow a required decision-making path.

该研究论文提出了一种自监督可解释概念嵌入模型（ICEMs），通过利用大型语言模型的泛化能力，在自监督方式下预测概念标签，并通过可解释的函数提供最终预测结果。实验结果表明，ICEMs 可以以自监督方式进行训练，并达到类似于全监督概念模型和端到端黑盒模型的性能。此外，该模型在预测方面具有解释性、互动性和可控性，能够提供有意义的逻辑解释、允许通过概念干预修改中间预测结果，并引导大型语言模型按照所需的决策路径进行解码。