Visual Dialog is a multimodal task of answering a sequence of questions
grounded in an image, using the conversation history as context. It entails
challenges in vision, language, reasoning, and grounding. However, studying
these subtasks in isolation on large, real datasets is infeasible as it
requires prohibitively-expensive complete annotation of the 'state' of all
images and dialogs.
We develop CLEVR-Dialog, a large diagnostic dataset for studying multi-round
reasoning in visual dialog. Specifically, we construct a dialog grammar that is
grounded in the scene graphs of the images from the CLEVR dataset. This
combination results in a dataset where all aspects of the visual dialog are
fully annotated. In total, CLEVR-Dialog contains 5 instances of 10-round
dialogs for about 85k CLEVR images, totaling to 4.25M question-answer pairs.
We use CLEVR-Dialog to benchmark performance of standard visual dialog
models; in particular, on visual coreference resolution (as a function of the
coreference distance). This is the first analysis of its kind for visual dialog
models that was not possible without this dataset. We hope the findings from
CLEVR-Dialog will help inform the development of future models for visual
dialog. Our dataset and code are publicly available.

开发了一个名为 CLEVR-Dialog 的大型诊断数据集，以研究视觉对话中的多轮推理（multiround reasoning），在其中所有视觉对话的所有方面都得到了完全注释。使用 CLEVR-Dialog 测量了标准视觉对话模型的性能，特别是针对视觉指代消解（coreference resolution）的表现，并于之前数据集中的模型进行了比较，发现了其中的差异。