Referential ambiguities arise in dialogue when a referring expression does
not uniquely identify the intended referent for the addressee. Addressees
usually detect such ambiguities immediately and work with the speaker to repair
it using meta-communicative, Clarificational Exchanges (CE): a Clarification
Request (CR) and a response. Here, we argue that the ability to generate and
respond to CRs imposes specific constraints on the architecture and objective
functions of multi-modal, visually grounded dialogue models. We use the SIMMC
2.0 dataset to evaluate the ability of different state-of-the-art model
architectures to process CEs, with a metric that probes the contextual updates
that arise from them in the model. We find that language-based models are able
to encode simple multi-modal semantic information and process some CEs,
excelling with those related to the dialogue history, whilst multi-modal models
can use additional learning objectives to obtain disentangled object
representations, which become crucial to handle complex referential ambiguities
across modalities overall.

在对话中，当引用表达不能唯一地识别出意图的指示对象时，产生了指向性模糊。此研究表明生成和回应澄清请求对于多模式、以视觉为基础的对话模型的架构和目标函数有特定的限制。通过使用 SIMMC 2.0 数据集评估不同最先进模型体系结构处理澄清交流的能力，该模型可探测它们在模型中引起的上下文更新的度量。结果发现，基于语言的模型能够编码简单的多模式语义信息和处理一些澄清交流，而多模式模型可以使用其他学习目标来获取解耦的物体表示，这对处理跨模态的复杂指向性模糊至关重要。