Resolving ambiguities through interaction is a hallmark of natural language,
and modeling this behavior is a core challenge in crafting AI assistants. In
this work, we study such behavior in LMs by proposing a task-agnostic framework
for resolving ambiguity by asking users clarifying questions. Our framework
breaks down this objective into three subtasks: (1) determining when
clarification is needed, (2) determining what clarifying question to ask, and
(3) responding accurately with the new information gathered through
clarification. We evaluate systems across three NLP applications: question
answering, machine translation and natural language inference. For the first
subtask, we present a novel uncertainty estimation approach, intent-sim, that
determines the utility of querying for clarification by estimating the entropy
over user intents. Our method consistently outperforms existing uncertainty
estimation approaches at identifying predictions that will benefit from
clarification. When only allowed to ask for clarification on 10% of examples,
our system is able to double the performance gains over randomly selecting
examples to clarify. Furthermore, we find that intent-sim is robust,
demonstrating improvements across a wide range of NLP tasks and LMs. Together,
our work lays foundation for studying clarifying interactions with LMs.

我们提出了一个任务无关的框架来通过询问用户澄清问题来解决模型中的歧义，我们的方法在确定何时需要澄清、确定要问的澄清问题以及通过澄清获得的新信息方面都取得了显著成果，并对模型的不确定性进行了估计，我们的方法在准确识别需要澄清的预测方面始终优于现有的不确定性估计方法，使系统能够在仅允许在 10％的示例上进行澄清时将性能提升两倍。