Text-based explanation is a particularly promising approach in explainable
AI, but the evaluation of text explanations is method-dependent. We argue that
placing the explanations on an information-theoretic framework could unify the
evaluations of two popular text explanation methods: rationale and natural
language explanations (NLE). This framework considers the post-hoc text
pipeline as a series of communication channels, which we refer to as
``explanation channels''. We quantify the information flow through these
channels, thereby facilitating the assessment of explanation characteristics.
We set up tools for quantifying two information scores: relevance and
informativeness. We illustrate what our proposed information scores measure by
comparing them against some traditional evaluation metrics. Our
information-theoretic scores reveal some unique observations about the
underlying mechanisms of two representative text explanations. For example, the
NLEs trade-off slightly between transmitting the input-related information and
the target-related information, whereas the rationales do not exhibit such a
trade-off mechanism. Our work contributes to the ongoing efforts in
establishing rigorous and standardized evaluation criteria in the rapidly
evolving field of explainable AI.

通过将解释信息放在信息论框架中，我们提出并量化了两种文本解释方法的信息流，并提供了对它们进行评估的工具和标准，从而为可解释的 AI 领域建立了严格和标准化的评估标准。