GuessWhat?! is a visual dialogue task between a guesser and an oracle. The guesser aims to locate an object supposed by the oracle oneself in an image by asking a sequence of Yes/No questions. Asking proper questions with the progress of dialogue is vital for achieving successful final guess. As a result, the progress of dialogue should be properly represented and tracked. Previous models for question generation pay less attention on the representation and tracking of dialogue states, and therefore are prone to asking low quality questions such as repeated questions. This paper proposes visual dialogue state tracking (VDST) based method for question generation. A visual dialogue state is defined as the distribution on objects in the image as well as representations of objects. Representations of objects are updated with the change of the distribution on objects. An object-difference based attention is used to decode new question. The distribution on objects is updated by comparing the question-answer pair and objects. Experimental results on GuessWhat?! dataset show that our model significantly outperforms existing methods and achieves new state-of-the-art performance. It is also noticeable that our model reduces the rate of repeated questions from more than 50% to 18.85% compared with previous state-of-the-art methods.

该论文提出了一种基于视觉对话状态跟踪（VDST）的方法，用于生成关于图片中目标的问题，该方法将对话状态表示和跟踪作为重点。该方法利用分布和对象差异注意力等特征来生成问题，并通过GuessWhat！数据集的实验结果证明其在减少重复问题等方面的表现明显优于先前的最先进方法。

视觉对话状态跟踪及其应用于问题生成