Transformer-based architectures have recently demonstrated remarkable
performance in the Visual Question Answering (VQA) task. However, such models
are likely to disregard crucial visual cues and often rely on multimodal
shortcuts and inherent biases of the language modality to predict the correct
answer, a phenomenon commonly referred to as lack of visual grounding. In this
work, we alleviate this shortcoming through a novel architecture for visual
question answering that leverages common sense reasoning as a supervisory
signal. Reasoning supervision takes the form of a textual justification of the
correct answer, with such annotations being already available on large-scale
Visual Common Sense Reasoning (VCR) datasets. The model's visual attention is
guided toward important elements of the scene through a similarity loss that
aligns the learned attention distributions guided by the question and the
correct reasoning. We demonstrate both quantitatively and qualitatively that
the proposed approach can boost the model's visual perception capability and
lead to performance increase, without requiring training on explicit grounding
annotations.

我们提出了一种新的视觉问答架构，通过常识推理作为监督信号来减轻模型在缺乏视觉基础的情况下的性能不足，并通过相似性损失将模型的视觉注意力引导到场景的重要元素，从而提高模型的视觉感知能力和性能。