The unprecedented performance of machine learning models in recent years,
particularly Deep Learning and transformer models, has resulted in their
application in various domains such as finance, healthcare, and education.
However, the models are error-prone and cannot be used autonomously, especially
in decision-making scenarios where, technically or ethically, the cost of error
is high. Moreover, because of the black-box nature of these models, it is
frequently difficult for the end user to comprehend the models' outcomes and
underlying processes to trust and use the model outcome to make a decision.
Explainable Artificial Intelligence (XAI) aids end-user understanding of the
model by utilizing approaches, including visualization techniques, to explain
and interpret the inner workings of the model and how it arrives at a result.
Although numerous research studies have been conducted recently focusing on the
performance of models and the XAI approaches, less work has been done on the
impact of explanations on human-AI team performance. This paper surveyed the
recent empirical studies on XAI's impact on human-AI decision-making,
identified the challenges, and proposed future research directions.

最近的实证研究调查了可解释人工智能对人工智能决策的影响，识别了挑战并提出未来的研究方向。

以解释为基础的决策：关于可解释人工智能在辅助决策中的角色的人类中心评估

Explain To Decide: A Human-Centric Review on the Role of Explainable  Artificial Intelligence in AI-assisted Decision Making

Many researchers motivate explainable AI with studies showing that human-AI
team performance on decision-making tasks improves when the AI explains its
recommendations. However, prior studies observed improvements from explanations
only when the AI, alone, outperformed both the human and the best team. Can
explanations help lead to complementary performance, where team accuracy is
higher than either the human or the AI working solo? We conduct mixed-method
user studies on three datasets, where an AI with accuracy comparable to humans
helps participants solve a task (explaining itself in some conditions). While
we observed complementary improvements from AI augmentation, they were not
increased by explanations. Rather, explanations increased the chance that
humans will accept the AI's recommendation, regardless of its correctness. Our
result poses new challenges for human-centered AI: Can we develop explanatory
approaches that encourage appropriate trust in AI, and therefore help generate
(or improve) complementary performance?

研究表明，仅当 AI 单独胜过人类和最佳团队时，AI 解释才能提高人类 - AI 团队在决策任务上的表现。本研究通过三个数据集上的混合法用户研究发现，AI 表现增强是有可能提高团队准确性的，但并非由于解释的作用，而是由于解释能提高人类对 AI 建议的接受率，进而鼓励恰当的信任和生成更好的绩效。