The field of eXplainable artificial intelligence (XAI) has produced a
plethora of methods (e.g., saliency-maps) to gain insight into artificial
intelligence (AI) models, and has exploded with the rise of deep learning (DL).
However, human-participant studies question the efficacy of these methods,
particularly when the AI output is wrong. In this study, we collected and
analyzed 156 human-generated text and saliency-based explanations collected in
a question-answering task (N=40) and compared them empirically to
state-of-the-art XAI explanations (integrated gradients, conservative LRP, and
ChatGPT) in a human-participant study (N=136). Our findings show that
participants found human saliency maps to be more helpful in explaining AI
answers than machine saliency maps, but performance negatively correlated with
trust in the AI model and explanations. This finding hints at the dilemma of AI
errors in explanation, where helpful explanations can lead to lower task
performance when they support wrong AI predictions.

本研究通过与最新方法（集成渐变、保守 LRP 和 ChatGPT）进行对比实证调查，收集和分析了 156 个人生成的文本和基于显著性的解释，并发现与机器显著性图相比，人工显著性图尤其在解释人工智能结果时更具说明性，但其性能与对人工智能模型和解释的信任负相关，揭示了人工智能解释中的困境：当有助于理解错误的人工智能预测时，会降低任务表现。