In order to interpret the communicative intents of an utterance, it needs to
be grounded in something that is outside of language; that is, grounded in
world modalities. In this paper, we argue that dialogue clarification
mechanisms make explicit the process of interpreting the communicative intents
of the speaker's utterances by grounding them in the various modalities in
which the dialogue is situated. This paper frames dialogue clarification
mechanisms as an understudied research problem and a key missing piece in the
giant jigsaw puzzle of natural language understanding. We discuss both the
theoretical background and practical challenges posed by this problem and
propose a recipe for obtaining grounding annotations. We conclude by
highlighting ethical issues that need to be addressed in future work.

本文提出对话澄清机制是自然语言理解中一个未被充分研究的关键问题，并讨论了该问题的理论背景、实际挑战和获取基础注释的方法。同时，还提出了需要在未来研究中解决的伦理问题。

基于场景的澄清标注配方

A recipe for annotating grounded clarifications

Current captioning approaches can describe images using black-box
architectures whose behavior is hardly controllable and explainable from the
exterior. As an image can be described in infinite ways depending on the goal
and the context at hand, a higher degree of controllability is needed to apply
captioning algorithms in complex scenarios. In this paper, we introduce a novel
framework for image captioning which can generate diverse descriptions by
allowing both grounding and controllability. Given a control signal in the form
of a sequence or set of image regions, we generate the corresponding caption
through a recurrent architecture which predicts textual chunks explicitly
grounded on regions, following the constraints of the given control.
Experiments are conducted on Flickr30k Entities and on COCO Entities, an
extended version of COCO in which we add grounding annotations collected in a
semi-automatic manner. Results demonstrate that our method achieves state of
the art performances on controllable image captioning, in terms of caption
quality and diversity. Code and annotations are publicly available at:
this https URL

本文介绍了一种新的图像字幕生成框架，该框架包括具有预测文本块和约束条件的递归架构，这些约束条件是通过控制信号（图像区域的序列或集合）确定的，并通过允许地面化和可控性来产生多样化的描述。实验表明，该方法在可控的图像字幕生成方面取得了最先进的性能。