Visual Dialog involves "understanding" the dialog history (what has been
discussed previously) and the current question (what is asked), in addition to
grounding information in the image, to generate the correct response. In this
paper, we show that co-attention models which explicitly encode dialog history
outperform models that don't, achieving state-of-the-art performance (72 % NDCG
on val set). However, we also expose shortcomings of the crowd-sourcing dataset
collection procedure by showing that history is indeed only required for a
small amount of the data and that the current evaluation metric encourages
generic replies. To that end, we propose a challenging subset (VisDialConv) of
the VisDial val set and provide a benchmark of 63% NDCG.

本文提出基于对话历史信息的注意力机制可以有效提高视觉对话系统的性能，并呈现出一种新的评估数据集进行基准测试，取得了 63% 的 NDCG 值。

视觉对话中的历史记录：我们真的需要吗？

History for Visual Dialog: Do we really need it?

Learning an effective attention mechanism for multimodal data is important in
many vision-and-language tasks that require a synergic understanding of both
the visual and textual contents. Existing state-of-the-art approaches use
co-attention models to associate each visual object (e.g., image region) with
each textual object (e.g., query word). Despite the success of these
co-attention models, they only model inter-modal interactions while neglecting
intra-modal interactions. Here we propose a general `unified attention' model
that simultaneously captures the intra- and inter-modal interactions of
multimodal features and outputs their corresponding attended representations.
By stacking such unified attention blocks in depth, we obtain the deep
Multimodal Unified Attention Network (MUAN), which can seamlessly be applied to
the visual question answering (VQA) and visual grounding tasks. We evaluate our
MUAN models on two VQA datasets and three visual grounding datasets, and the
results show that MUAN achieves top-level performance on both tasks without
bells and whistles.

本研究提出了一种统一的注意力模型，能够同时捕捉多模态特征的内部和跨模态之间的相互作用，并输出相应的注意力表示。通过堆叠这样的统一注意力块，我们获得了深度多模态统一注意力网络 (MUAN)，可以无缝地应用于视觉问答 (VQA) 和视觉定位任务。通过在两个 VQA 数据集和三个视觉定位数据集上评估 MUAN 模型，结果表明 MUAN 在两个任务上都实现了顶级性能。