In high-stakes domains like medical reasoning, AI assistants powered by large
language models (LLMs) are yet to be reliable and safe. We identify a key
obstacle towards reliability: existing LLMs are trained to answer any question,
even with incomplete context in the prompt or insufficient parametric
knowledge. We propose to change this paradigm to develop more careful LLMs that
ask follow-up questions to gather necessary and sufficient information and
respond reliably. We introduce MEDIQ, a framework to simulate realistic
clinical interactions, which incorporates a Patient System and an adaptive
Expert System. The Patient may provide incomplete information in the beginning;
the Expert refrains from making diagnostic decisions when unconfident, and
instead elicits missing details from the Patient via follow-up questions. To
evaluate MEDIQ, we convert MEDQA and CRAFT-MD -- medical benchmarks for
diagnostic question answering -- into an interactive setup. We develop a
reliable Patient system and prototype several Expert systems, first showing
that directly prompting state-of-the-art LLMs to ask questions degrades the
quality of clinical reasoning, indicating that adapting LLMs to interactive
information-seeking settings is nontrivial. We then augment the Expert with a
novel abstention module to better estimate model confidence and decide whether
to ask more questions, thereby improving diagnostic accuracy by 20.3%; however,
performance still lags compared to an (unrealistic in practice) upper bound
when full information is given upfront. Further analyses reveal that
interactive performance can be improved by filtering irrelevant contexts and
reformatting conversations. Overall, our paper introduces a novel problem
towards LLM reliability, a novel MEDIQ framework, and highlights important
future directions to extend the information-seeking abilities of LLM assistants
in critical domains.

在高风险领域如医疗推理中，目前大语言模型的 AI 助理尚缺乏可靠性和安全性。本文提出了挑战可靠性的主要障碍：现有的大语言模型被训练以回答任何问题，即使在提问时存在不完整的上下文或不足的参数知识。我们建议通过开发更加谨慎的大语言模型来改变这种模式，让它们能够通过追问问题来收集必要和充分的信息并提供可靠的回答。我们引入了 MEDIQ 框架，用于模拟逼真的临床交互过程，其中包括患者系统和自适应专家系统。患者系统可能在开始阶段提供不完整的信息；当专家系统对决策缺乏信心时，会通过追问问题来获取患者缺失的详细信息，而不会做出诊断决策。为了评估 MEDIQ，我们将 MEDQA 和 CRAFT-MD（用于诊断问题回答的医学基准测试）转化为交互式设置。我们开发了一个可靠的患者系统，并原型化了几个专家系统，首先表明直接命令最先进的大语言模型提问会降低临床推理的质量，表明将大语言模型适应于交互式信息寻求环境是困难的。然后，我们使用新的弃权模型来改进专家系统，更好地估计模型置信度，并决定是否追问更多问题，从而将诊断准确率提高了 20.3%；然而，性能仍然落后于完整信息均提供的理论上限（在实践中不切实际）。进一步分析表明，通过过滤不相关的上下文和重新格式化对话，可以提高交互性能。总体而言，我们的论文介绍了一个关于大语言模型可靠性的新问题，一个新的 MEDIQ 框架，并强调了在关键领域扩展大语言模型助理的信息寻求能力的重要未来方向。