The current literature on AI-advised decision making -- involving explainable
AI systems advising human decision makers -- presents a series of inconclusive
and confounding results. To synthesize these findings, we propose a simple
theory that elucidates the frequent failure of AI explanations to engender
appropriate reliance and complementary decision making performance. We argue
explanations are only useful to the extent that they allow a human decision
maker to verify the correctness of an AI's prediction, in contrast to other
desiderata, e.g., interpretability or spelling out the AI's reasoning process.
Prior studies find in many decision making contexts AI explanations do not
facilitate such verification. Moreover, most contexts fundamentally do not
allow verification, regardless of explanation method. We conclude with a
discussion of potential approaches for more effective explainable AI-advised
decision making and human-AI collaboration.

本文聚焦 AI 辅助决策，在 AI 解释人类决策过程中频繁失败的背景下，提出了一个简单的理论，即解释只有在允许决策者验证其正确性时才有用，我们讨论了更有效的 AI 决策解释方法和人工智能 - 人类决策的协作。

寻求可验证性：解释很少能够在 AI 辅助的决策判断中提供补充性的表现

In Search of Verifiability: Explanations Rarely Enable Complementary  Performance in AI-Advised Decision Making

AI practitioners typically strive to develop the most accurate systems,
making an implicit assumption that the AI system will function autonomously.
However, in practice, AI systems often are used to provide advice to people in
domains ranging from criminal justice and finance to healthcare. In such
AI-advised decision making, humans and machines form a team, where the human is
responsible for making final decisions. But is the most accurate AI the best
teammate? We argue "No" -- predictable performance may be worth a slight
sacrifice in AI accuracy. Instead, we argue that AI systems should be trained
in a human-centered manner, directly optimized for team performance. We study
this proposal for a specific type of human-AI teaming, where the human overseer
chooses to either accept the AI recommendation or solve the task themselves. To
optimize the team performance for this setting we maximize the team's expected
utility, expressed in terms of the quality of the final decision, cost of
verifying, and individual accuracies of people and machines. Our experiments
with linear and non-linear models on real-world, high-stakes datasets show that
the most accuracy AI may not lead to highest team performance and show the
benefit of modeling teamwork during training through improvements in expected
team utility across datasets, considering parameters such as human skill and
the cost of mistakes. We discuss the shortcoming of current optimization
approaches beyond well-studied loss functions such as log-loss, and encourage
future work on AI optimization problems motivated by human-AI collaboration.

我们应该训练 AI 系统以人为中心的方式来最大化人工智能与人类决策者团队的绩效表现，并通过实验研究来证明，最高准确率的 AI 模型并不一定能够带来最优团队绩效。