The field of eXplainable artificial intelligence (XAI) has produced a
plethora of methods (e.g., saliency-maps) to gain insight into artificial
intelligence (AI) models, and has exploded with the rise of deep learning (DL).
However, human-participant studies question the efficacy of these methods,
particularly when the AI output is wrong. In this study, we collected and
analyzed 156 human-generated text and saliency-based explanations collected in
a question-answering task (N=40) and compared them empirically to
state-of-the-art XAI explanations (integrated gradients, conservative LRP, and
ChatGPT) in a human-participant study (N=136). Our findings show that
participants found human saliency maps to be more helpful in explaining AI
answers than machine saliency maps, but performance negatively correlated with
trust in the AI model and explanations. This finding hints at the dilemma of AI
errors in explanation, where helpful explanations can lead to lower task
performance when they support wrong AI predictions.

本研究通过与最新方法（集成渐变、保守 LRP 和 ChatGPT）进行对比实证调查，收集和分析了 156 个人生成的文本和基于显著性的解释，并发现与机器显著性图相比，人工显著性图尤其在解释人工智能结果时更具说明性，但其性能与对人工智能模型和解释的信任负相关，揭示了人工智能解释中的困境：当有助于理解错误的人工智能预测时，会降低任务表现。

解开人工智能错误之谜：探索大型语言模型的人工和机器解释的有效性

Unraveling the Dilemma of AI Errors: Exploring the Effectiveness of  Human and Machine Explanations for Large Language Models

We present a new methodology for handling AI errors by introducing weakly
supervised AI error correctors with a priori performance guarantees. These AI
correctors are auxiliary maps whose role is to moderate the decisions of some
previously constructed underlying classifier by either approving or rejecting
its decisions. The rejection of a decision can be used as a signal to suggest
abstaining from making a decision. A key technical focus of the work is in
providing performance guarantees for these new AI correctors through bounds on
the probabilities of incorrect decisions. These bounds are distribution
agnostic and do not rely on assumptions on the data dimension. Our empirical
example illustrates how the framework can be applied to improve the performance
of an image classifier in a challenging real-world task where training data are
scarce.

通过引入具有先验性能保证的弱监督的 AI 错误修正器，我们提出了一种处理 AI 错误的新方法。这些 AI 修正器是辅助映射，其作用是通过批准或拒绝某些先前构建的基础分类器的决策来调节其决策。拒绝决策可以用作建议放弃做出决策的信号。该工作的关键技术重点是通过对错误决策概率的界限来提供这些新的 AI 修正器的性能保证。这些界限是分布无关的，不依赖于对数据维度的假设。我们的实证示例说明了该框架如何应用于改善训练数据稀缺的具有挑战性的现实任务中的图像分类器的性能。