Visual question answering (VQA) is a critical multimodal task in which an
agent must answer questions according to the visual cue. Unfortunately,
language bias is a common problem in VQA, which refers to the model generating
answers only by associating with the questions while ignoring the visual
content, resulting in biased results. We tackle the language bias problem by
proposing a self-supervised counterfactual metric learning (SC-ML) method to
focus the image features better. SC-ML can adaptively select the
question-relevant visual features to answer the question, reducing the negative
influence of question-irrelevant visual features on inferring answers. In
addition, question-irrelevant visual features can be seamlessly incorporated
into counterfactual training schemes to further boost robustness. Extensive
experiments have proved the effectiveness of our method with improved results
on the VQA-CP dataset. Our code will be made publicly available.

我们提出了一种自我监督的反事实度量学习方法来应对 VQA 中的语言偏见问题，该方法可以自适应地选择与问题相关的视觉特征来回答问题，从而减少与问题不相关的视觉特征对推断答案的负面影响，实验证明了我们方法对 VQA-CP 数据集具有提升效果。