GQA~\citep{hudson2019gqa} is a dataset for real-world visual reasoning and
compositional question answering. We found that many answers predicted by the
best vision-language models on the GQA dataset do not match the ground-truth
answer but still are semantically meaningful and correct in the given context.
In fact, this is the case with most existing visual question answering (VQA)
datasets where they assume only one ground-truth answer for each question. We
propose Alternative Answer Sets (AAS) of ground-truth answers to address this
limitation, which is created automatically using off-the-shelf NLP tools. We
introduce a semantic metric based on AAS and modify top VQA solvers to support
multiple plausible answers for a question. We implement this approach on the
GQA dataset and show the performance improvements. Code and data are available
in this link https://github.com/luomancs/alternative_answer_set.git.

提出了基于 Alternative Answer Sets (AAS) 的语义度量方法，通过修改现有的 VQA 算法实现多个有效答案，提高了视觉理解和组合问题回答 (GQA) 数据集的表现。

开放式视觉问题回答（VQA）任务开发和评估中的瓶颈突破：“你对并不代表我错

'Just because you are right, doesn't mean I am wrong': Overcoming a  Bottleneck in the Development and Evaluation of Open-Ended Visual Question  Answering (VQA) Tasks

With the development of deep learning techniques and large scale datasets,
the question answering (QA) systems have been quickly improved, providing more
accurate and satisfying answers. However, current QA systems either focus on
the sentence-level answer, i.e., answer selection, or phrase-level answer,
i.e., machine reading comprehension. How to produce compositional answers has
not been throughout investigated. In compositional question answering, the
systems should assemble several supporting evidence from the document to
generate the final answer, which is more difficult than sentence-level or
phrase-level QA. In this paper, we present a large-scale compositional question
answering dataset containing more than 120k human-labeled questions. The answer
in this dataset is composed of discontiguous sentences in the corresponding
document. To tackle the ComQA problem, we proposed a hierarchical graph neural
networks, which represents the document from the low-level word to the
high-level sentence. We also devise a question selection and node selection
task for pre-training. Our proposed model achieves a significant improvement
over previous machine reading comprehension methods and pre-training methods.
Codes and dataset can be found at https://github.com/benywon/ComQA.

本研究提出了一个包含超过 120k 个人工标注问题的大规模组合式问答数据集，回答在此数据集上的问题需要组合文档中的若干段落来生成最终答案。本文针对该问题提出了一种基于图神经网络的层次预训练模型，并在预训练和机器阅读理解领域中取得了巨大的提升。