Deep learning models exhibit a preference for statistical fitting over
logical reasoning. Spurious correlations might be memorized when there exists
statistical bias in training data, which severely limits the model performance
especially in small data scenarios. In this work, we introduce Counterfactual
Adversarial Training framework (CAT) to tackle the problem from a causality
perspective. Particularly, for a specific sample, CAT first generates a
counterfactual representation through latent space interpolation in an
adversarial manner, and then performs Counterfactual Risk Minimization (CRM) on
each original-counterfactual pair to adjust sample-wise loss weight
dynamically, which encourages the model to explore the true causal effect.
Extensive experiments demonstrate that CAT achieves substantial performance
improvement over SOTA across different downstream tasks, including sentence
classification, natural language inference and question answering.

本文提出了一个基于因果关系的 Counterfactual Adversarial Training 框架（CAT），通过对单个样本生成对应的反事实表示，动态调整样本特有的损失权重，以此鼓励模型探索真实的因果关系，实现在句子分类、自然语言推理和问答等领域的显著性能提升。