Widely deployed large language models (LLMs) can produce convincing yet
incorrect outputs, potentially misleading users who may rely on them as if they
were correct. To reduce such overreliance, there have been calls for LLMs to
communicate their uncertainty to end users. However, there has been little
empirical work examining how users perceive and act upon LLMs' expressions of
uncertainty. We explore this question through a large-scale, pre-registered,
human-subject experiment (N=404) in which participants answer medical questions
with or without access to responses from a fictional LLM-infused search engine.
Using both behavioral and self-reported measures, we examine how different
natural language expressions of uncertainty impact participants' reliance,
trust, and overall task performance. We find that first-person expressions
(e.g., "I'm not sure, but...") decrease participants' confidence in the system
and tendency to agree with the system's answers, while increasing participants'
accuracy. An exploratory analysis suggests that this increase can be attributed
to reduced (but not fully eliminated) overreliance on incorrect answers. While
we observe similar effects for uncertainty expressed from a general perspective
(e.g., "It's not clear, but..."), these effects are weaker and not
statistically significant. Our findings suggest that using natural language
expressions of uncertainty may be an effective approach for reducing
overreliance on LLMs, but that the precise language used matters. This
highlights the importance of user testing before deploying LLMs at scale.

广泛应用的大型语言模型（LLM）能够生成具有说服力但不正确的结果，存在误导用户的风险。为减少过度依赖，有呼吁要求 LLM 向最终用户传达其不确定性。然而，有关用户如何感知和行动 LLM 的不确定表达的实证研究很少。通过大规模的、预先注册的人体实验（N=404），我们探索了这个问题，参与者回答医学问题，带或不带来自一个虚构的 LLM 增强搜索引擎的回复。通过行为和自我报告的测量，我们研究了不同的自然语言不确定表达方式对参与者信赖、信任和任务整体表现的影响。我们发现第一人称表达（如 “我不确定，但是...”）降低了参与者对系统的信心和同意系统答案的倾向，同时提高了参与者的准确性。初步分析表明，这种增加可以归因于对错误答案的减少（但并非完全消除）的过度依赖。虽然我们观察到了不确定性从一般角度表达的类似效果（如 “不清楚，但是...”），但这些效果较弱且不具有统计学意义。我们的发现表明，使用自然语言的不确定表达可能是减少对 LLM 过度依赖的有效方法，但使用的确切语言很重要。这凸显了在部署大规模 LLM 之前进行用户测试的重要性。

大型语言模型的不确定表达对用户依赖和信任的影响研究

"I'm Not Sure, But...": Examining the Impact of Large Language Models'  Uncertainty Expression on User Reliance and Trust

As the popularity of adhesive joints in industry increases, so does the need
for tools to support the process of selecting a suitable adhesive. While some
such tools already exist, they are either too limited in scope, or offer too
little flexibility in use. This work presents a more advanced tool, that was
developed together with a team of adhesive experts. We first extract the
experts' knowledge about this domain and formalize it in a Knowledge Base (KB).
The IDP-Z3 reasoning system can then be used to derive the necessary
functionality from this KB. Together with a user-friendly interactive
interface, this creates an easy-to-use tool capable of assisting the adhesive
experts. To validate our approach, we performed user testing in the form of
qualitative interviews. The experts are very positive about the tool, stating
that, among others, it will help save time and find more suitable adhesives.
Under consideration in Theory and Practice of Logic Programming (TPLP).

通过与粘接专家团队的合作，本研究提出了一种高级工具，包括一个知识库和交互界面，能够协助粘接专家节省时间并找到更合适的胶粘剂。