Growing interest in conversational agents promote twoway human-computer
communications involving asking and answering visual questions have become an
active area of research in AI. Thus, generation of visual questionanswer
pair(s) becomes an important and challenging task. To address this issue, we
propose a weakly-supervised visual question answer generation method that
generates a relevant question-answer pairs for a given input image and
associated caption. Most of the prior works are supervised and depend on the
annotated question-answer datasets. In our work, we present a weakly supervised
method that synthetically generates question-answer pairs procedurally from
visual information and captions. The proposed method initially extracts list of
answer words, then does nearest question generation that uses the caption and
answer word to generate synthetic question. Next, the relevant question
generator converts the nearest question to relevant language question by
dependency parsing and in-order tree traversal, finally, fine-tune a ViLBERT
model with the question-answer pair(s) generated at end. We perform an
exhaustive experimental analysis on VQA dataset and see that our model
significantly outperform SOTA methods on BLEU scores. We also show the results
wrt baseline models and ablation study.

本文提出一种弱监督视觉问答生成方法，从视觉信息和字幕中合成问答对，使用 ViLBERT 模型对其进行微调，实验结果在 VQA 数据集上表现显著优于其他先进技术。

弱监督视觉问答生成

Weakly Supervised Visual Question Answer Generation

Question and answer generation (QAG) consists of generating a set of
question-answer pairs given a context (e.g. a paragraph). This task has a
variety of applications, such as data augmentation for question answering (QA)
models, information retrieval and education. In this paper, we establish
baselines with three different QAG methodologies that leverage
sequence-to-sequence language model (LM) fine-tuning. Experiments show that an
end-to-end QAG model, which is computationally light at both training and
inference times, is generally robust and outperforms other more convoluted
approaches. However, there are differences depending on the underlying
generative LM. Finally, our analysis shows that QA models fine-tuned solely on
generated question-answer pairs can be competitive when compared to supervised
QA models trained on human-labeled data.

本文旨在建立三种不同的 QAG 方法的基线，利用序列到序列语言模型进行微调。实验证明，一种端到端的 QAG 模型，它在训练和推理时计算量都很轻，通常是稳健的，并且胜过其他复杂的方法。然而，这取决于底层生成模型的不同。最后，我们的分析表明，仅基于生成的问答对微调的 QA 模型可以与人类标注数据上训练的监督 QA 模型具有竞争力。