Grounding a pronoun to a visual object it refers to requires complex
reasoning from various information sources, especially in conversational
scenarios. For example, when people in a conversation talk about something all
speakers can see, they often directly use pronouns (e.g., it) to refer to it
without previous introduction. This fact brings a huge challenge for modern
natural language understanding systems, particularly conventional context-based
pronoun coreference models. To tackle this challenge, in this paper, we
formally define the task of visual-aware pronoun coreference resolution (PCR)
and introduce VisPro, a large-scale dialogue PCR dataset, to investigate
whether and how the visual information can help resolve pronouns in dialogues.
We then propose a novel visual-aware PCR model, VisCoref, for this task and
conduct comprehensive experiments and case studies on our dataset. Results
demonstrate the importance of the visual information in this PCR case and show
the effectiveness of the proposed model.

本文讨论了在会话中处理代词回指的复杂性，并提出一种基于视觉信息的模型来解决这一问题。经过对提出模型的数据集进行对比试验，证明了视觉信息在代词回指中的重要性和所提出模型的有效性。

所见即所得：对话中视觉代词共指消解

What You See is What You Get: Visual Pronoun Coreference Resolution in  Dialogues

Resolving pronoun coreference requires knowledge support, especially for
particular domains (e.g., medicine). In this paper, we explore how to leverage
different types of knowledge to better resolve pronoun coreference with a
neural model. To ensure the generalization ability of our model, we directly
incorporate knowledge in the format of triplets, which is the most common
format of modern knowledge graphs, instead of encoding it with features or
rules as that in conventional approaches. Moreover, since not all knowledge is
helpful in certain contexts, to selectively use them, we propose a knowledge
attention module, which learns to select and use informative knowledge based on
contexts, to enhance our model. Experimental results on two datasets from
different domains prove the validity and effectiveness of our model, where it
outperforms state-of-the-art baselines by a large margin. Moreover, since our
model learns to use external knowledge rather than only fitting the training
data, it also demonstrates superior performance to baselines in the
cross-domain setting.

本文探索了如何利用不同类型的知识图谱来更好地解决代词共参问题，并提出了一种知识注意力模块，以选择性地使用信息化知识来增强模型，实验结果表明，在不同领域的两个数据集上，我们的模型优于现有技术，并在跨领域的情况下也表现出更好的性能。