AI explanations are often mentioned as a way to improve human-AI
decision-making. Yet, empirical studies have not found consistent evidence of
explanations' effectiveness and, on the contrary, suggest that they can
increase overreliance when the AI system is wrong. While many factors may
affect reliance on AI support, one important factor is how decision-makers
reconcile their own intuition -- which may be based on domain knowledge, prior
task experience, or pattern recognition -- with the information provided by the
AI system to determine when to override AI predictions. We conduct a
think-aloud, mixed-methods study with two explanation types (feature- and
example-based) for two prediction tasks to explore how decision-makers'
intuition affects their use of AI predictions and explanations, and ultimately
their choice of when to rely on AI. Our results identify three types of
intuition involved in reasoning about AI predictions and explanations:
intuition about the task outcome, features, and AI limitations. Building on
these, we summarize three observed pathways for decision-makers to apply their
own intuition and override AI predictions. We use these pathways to explain why
(1) the feature-based explanations we used did not improve participants'
decision outcomes and increased their overreliance on AI, and (2) the
example-based explanations we used improved decision-makers' performance over
feature-based explanations and helped achieve complementary human-AI
performance. Overall, our work identifies directions for further development of
AI decision-support systems and explanation methods that help decision-makers
effectively apply their intuition to achieve appropriate reliance on AI.

此研究探讨了决策者对于任务预测和解释的一些直觉，发现了三种类型的直觉：关于任务结果、特征和 AI 限制。此外，该研究还通过两种类型的解释，探索了决策者如何运用自己的直觉来决定是否应该依赖 AI 预测，并发现了三条决策者运用自己直觉的路径。研究表明，AI 解释并不能总是提高决策者的决策效果并可能提高其对 AI 的过度依赖，但使用基于示例的解释则可能有助于完善人工智能的表现。

人的直觉在人工智能决策中的依赖作用及其解释

Understanding the Role of Human Intuition on Reliance in Human-AI Decision-Making with Explanations

Prior work has identified a resilient phenomenon that threatens the
performance of human-AI decision-making teams: overreliance, when people agree
with an AI, even when it is incorrect. Surprisingly, overreliance does not
reduce when the AI produces explanations for its predictions, compared to only
providing predictions. Some have argued that overreliance results from
cognitive biases or uncalibrated trust, attributing overreliance to an
inevitability of human cognition. By contrast, our paper argues that people
strategically choose whether or not to engage with an AI explanation,
demonstrating empirically that there are scenarios where AI explanations reduce
overreliance. To achieve this, we formalize this strategic choice in a
cost-benefit framework, where the costs and benefits of engaging with the task
are weighed against the costs and benefits of relying on the AI. We manipulate
the costs and benefits in a maze task, where participants collaborate with a
simulated AI to find the exit of a maze. Through 5 studies (N = 731), we find
that costs such as task difficulty (Study 1), explanation difficulty (Study 2,
3), and benefits such as monetary compensation (Study 4) affect overreliance.
Finally, Study 5 adapts the Cognitive Effort Discounting paradigm to quantify
the utility of different explanations, providing further support for our
framework. Our results suggest that some of the null effects found in
literature could be due in part to the explanation not sufficiently reducing
the costs of verifying the AI's prediction.

本文探讨了人类和人工智能决策团队的过度依赖现象，提出了一种基于成本效益框架的策略模型来平衡 AI 解释带来的成本和效益，证明了在某些情况下 AI 解释可降低过度依赖。