Most existing works in visual question answering (VQA) are dedicated to
improving the accuracy of predicted answers, while disregarding the
explanations. We argue that the explanation for an answer is of the same or
even more importance compared with the answer itself, since it makes the
question and answering process more understandable and traceable. To this end,
we propose a new task of VQA-E (VQA with Explanation), where the computational
models are required to generate an explanation with the predicted answer. We
first construct a new dataset, and then frame the VQA-E problem in a multi-task
learning architecture. Our VQA-E dataset is automatically derived from the VQA
v2 dataset by intelligently exploiting the available captions. We have
conducted a user study to validate the quality of explanations synthesized by
our method. We quantitatively show that the additional supervision from
explanations can not only produce insightful textual sentences to justify the
answers, but also improve the performance of answer prediction. Our model
outperforms the state-of-the-art methods by a clear margin on the VQA v2
dataset.

提出了 VQA-E 任务，要求计算机模型在预测答案的同时生成一个解释。通过多任务学习框架， VQA-E 数据集从 VQA v2 数据集自动导出，用户研究表明，我们的方法可以生成有洞察力的文本句子来证明答案，并提高了答案预测的性能。