Counterfactually-Augmented Data (CAD) -- minimal editing of sentences to flip
the corresponding labels -- has the potential to improve the
Out-Of-Distribution (OOD) generalization capability of language models, as CAD
induces language models to exploit domain-independent causal features and
exclude spurious correlations. However, the empirical results of CAD's OOD
generalization are not as efficient as anticipated. In this study, we attribute
the inefficiency to the myopia phenomenon caused by CAD: language models only
focus on causal features that are edited in the augmentation operation and
exclude other non-edited causal features. Therefore, the potential of CAD is
not fully exploited. To address this issue, we analyze the myopia phenomenon in
feature space from the perspective of Fisher's Linear Discriminant, then we
introduce two additional constraints based on CAD's structural properties
(dataset-level and sentence-level) to help language models extract more
complete causal features in CAD, thereby mitigating the myopia phenomenon and
improving OOD generalization capability. We evaluate our method on two tasks:
Sentiment Analysis and Natural Language Inference, and the experimental results
demonstrate that our method could unlock the potential of CAD and improve the
OOD generalization performance of language models by 1.0% to 5.9%.

通过引入两个额外的基于 Counterfactually-Augmented Data 的结构属性约束，从而从 Fisher 的线性判别的角度分析特征空间中的近视现象，本研究发现 Counterfactually-Augmented Data 的潜力没有被充分利用，通过改进的方法，可以提高语言模型的 OOD 泛化能力，进而提高模型性能。

释放反事实增强数据在超出分布的泛化中的潜力

Unlock the Potential of Counterfactually-Augmented Data in  Out-Of-Distribution Generalization

Counterfactually-Augmented Data (CAD) has the potential to improve language
models' Out-Of-Distribution (OOD) generalization capability, as CAD induces
language models to exploit causal features and exclude spurious correlations.
However, the empirical results of OOD generalization on CAD are not as
efficient as expected. In this paper, we attribute the inefficiency to Myopia
Phenomenon caused by CAD: language models only focus on causal features that
are edited in the augmentation and exclude other non-edited causal features. As
a result, the potential of CAD is not fully exploited. Based on the structural
properties of CAD, we design two additional constraints to help language models
extract more complete causal features contained in CAD, thus improving the OOD
generalization capability. We evaluate our method on two tasks: Sentiment
Analysis and Natural Language Inference, and the experimental results
demonstrate that our method could unlock CAD's potential and improve language
models' OOD generalization capability.

本文研究了 Counterfactually-Augmented Data 对自然语言处理模型的改进作用，发现其普遍存在的近视现象导致了其潜能无法完全被发掘。作者设计了两类新的约束来帮助提取更完整的因果特征，通过情感分析和自然语言推理两个任务来验证该方法，实验结果表明该方法能够释放 CAD 的潜能，并提高模型的 OOD 泛化能力。