Transformer architectures are complex and their use in NLP, while it has engendered many successes, makes their interpretability or explainability challenging. Recent debates have shown that attention maps and attribution methods are unreliable (Pruthi et al., 2019; Brunner et al., 2019). In this paper, we present some of their limitations and introduce COCKATIEL, which successfully addresses some of them. COCKATIEL is a novel, post-hoc, concept-based, model-agnostic XAI technique that generates meaningful explanations from the last layer of a neural net model trained on an NLP classification task by using Non-Negative Matrix Factorization (NMF) to discover the concepts the model leverages to make predictions and by exploiting a Sensitivity Analysis to estimate accurately the importance of each of these concepts for the model. It does so without compromising the accuracy of the underlying model or requiring a new one to be trained. We conduct experiments in single and multi-aspect sentiment analysis tasks and we show COCKATIEL's superior ability to discover concepts that align with humans' on Transformer models without any supervision, we objectively verify the faithfulness of its explanations through fidelity metrics, and we showcase its ability to provide meaningful explanations in two different datasets.

本文介绍了 COCKATIEL 技术，它是一种后置的、基于概念的、模型无关的 XAI 技术，使用 NMF 和敏感性分析从 NLP 分类任务的神经网络模型的最后一层生成有意义的解释，无需牺牲模型准确性或训练新模型，并在单方面和多方面情感分析任务中展示其优越的能力。

COCKATIEL：一种用于NLP任务神经网络分类器解释的连续概念排名可解释元素的属性方法