Clarification questions are an essential dialogue tool to signal
misunderstanding, ambiguities, and under-specification in language use. While
humans are able to resolve uncertainty by asking questions since childhood,
modern dialogue systems struggle to generate effective questions. To make
progress in this direction, in this work we take a collaborative dialogue task
as a testbed and study how model uncertainty relates to human uncertainty -- an
as yet under-explored problem. We show that model uncertainty does not mirror
human clarification-seeking behavior, which suggests that using human
clarification questions as supervision for deciding when to ask may not be the
most effective way to resolve model uncertainty. To address this issue, we
propose an approach to generating clarification questions based on model
uncertainty estimation, compare it to several alternatives, and show that it
leads to significant improvements in terms of task success. Our findings
highlight the importance of equipping dialogue systems with the ability to
assess their own uncertainty and exploit in interaction.

通过团队对话任务为实验平台，研究模型不确定性与人类不确定性之间的关系，提出了一种基于模型不确定性估计的生成澄清问题的方法，并与其他几种方法进行对比，显示出在任务成功方面有显著的改进。

在适当时候提出正确问题：人类和模型不确定性对于提问澄清问题的指导

Asking the Right Question at the Right Time: Human and Model Uncertainty  Guidance to Ask Clarification Questions

Contemporary vision benchmarks predominantly consider tasks on which humans
can achieve near-perfect performance. However, humans are frequently presented
with visual data that they cannot classify with 100% certainty, and models
trained on standard vision benchmarks achieve low performance when evaluated on
this data. To address this issue, we introduce a procedure for creating
datasets of ambiguous images and use it to produce SQUID-E ("Squidy"), a
collection of noisy images extracted from videos. All images are annotated with
ground truth values and a test set is annotated with human uncertainty
judgments. We use this dataset to characterize human uncertainty in vision
tasks and evaluate existing visual event classification models. Experimental
results suggest that existing vision models are not sufficiently equipped to
provide meaningful outputs for ambiguous images and that datasets of this
nature can be used to assess and improve such models through model training and
direct evaluation of model calibration. These findings motivate large-scale
ambiguous dataset creation and further research focusing on noisy visual data.

本文提出了一种制作模糊图像数据集的方法，并使用此数据集评估现有视觉事件分类模型。实验结果表明，现有的视觉模型无法针对模糊图像提供有意义的输出，因此需要更多类似模糊图像数据集的创造和模型的研究。