Today's VQA models still tend to capture superficial linguistic correlations
in the training set and fail to generalize to the test set with different QA
distributions. To reduce these language biases, recent VQA works introduce an
auxiliary question-only model to regularize the training of targeted VQA model,
and achieve dominating performance on diagnostic benchmarks for
out-of-distribution testing. However, due to complex model design, these
ensemble-based methods are unable to equip themselves with two indispensable
characteristics of an ideal VQA model: 1) Visual-explainable: The model should
rely on the right visual regions when making decisions. 2) Question-sensitive:
The model should be sensitive to the linguistic variations in questions. To
this end, we propose a novel model-agnostic Counterfactual Samples Synthesizing
and Training (CSST) strategy. After training with CSST, VQA models are forced
to focus on all critical objects and words, which significantly improves both
visual-explainable and question-sensitive abilities. Specifically, CSST is
composed of two parts: Counterfactual Samples Synthesizing (CSS) and
Counterfactual Samples Training (CST). CSS generates counterfactual samples by
carefully masking critical objects in images or words in questions and
assigning pseudo ground-truth answers. CST not only trains the VQA models with
both complementary samples to predict respective ground-truth answers, but also
urges the VQA models to further distinguish the original samples and
superficially similar counterfactual ones. To facilitate the CST training, we
propose two variants of supervised contrastive loss for VQA, and design an
effective positive and negative sample selection mechanism based on CSS.
Extensive experiments have shown the effectiveness of CSST. Particularly, by
building on top of model LMH+SAR, we achieve record-breaking performance on all
OOD benchmarks.

介绍了一个新的模型 - 不可知模型 Counterfactual Samples Synthesizing and Training（CSST）模型，可以消除 VQA 模型的语言偏见，并使模型变得更加视觉可解释和对语言变化更敏感，并在所有 OOD 基准测试上取得了创纪录的表现。