While models for Visual Question Answering (VQA) have steadily improved over the years, interacting with one quickly reveals that these models lack consistency. For instance, if a model answers "red" to "What color is the balloon?", it might answer "no" if asked, "Is the balloon red?". These responses violate simple notions of entailment and raise questions about how effectively VQA models ground language. In this work, we introduce a dataset, ConVQA, and metrics that enable quantitative evaluation of consistency in VQA. For a given observable fact in an image (e.g. the balloon's color), we generate a set of logically consistent question-answer (QA) pairs (e.g. Is the balloon red?) and also collect a human-annotated set of common-sense based consistent QA pairs (e.g. Is the balloon the same color as tomato sauce?). Further, we propose a consistency-improving data augmentation module, a Consistency Teacher Module (CTM). CTM automatically generates entailed (or similar-intent) questions for a source QA pair and fine-tunes the VQA model if the VQA's answer to the entailed question is consistent with the source QA pair. We demonstrate that our CTM-based training improves the consistency of VQA models on the ConVQA datasets and is a strong baseline for further research.

本研究提出了一种方法，通过引入一个名为ConVQA的数据集和度量标准，量化衡量视觉问答（VQA）模型的一致性，并建立了一种名为一致性教师模块（CTM）的数据增强模块，该模块可以自动增强源QA对的语义相关问题，从而优化VQA的一致性。在ConVQA数据集上的实验结果表明，我们的方法可以提高现有VQA模型的一致性表现。

通过导出问题生成改善VQA中的回答一致性