As NLP models are increasingly deployed in socially situated settings such as
online abusive content detection, it is crucial to ensure that these models are
robust. One way of improving model robustness is to generate counterfactually
augmented data (CAD) for training models that can better learn to distinguish
between core features and data artifacts. While models trained on this type of
data have shown promising out-of-domain generalizability, it is still unclear
what the sources of such improvements are. We investigate the benefits of CAD
for social NLP models by focusing on three social computing constructs --
sentiment, sexism, and hate speech. Assessing the performance of models trained
with and without CAD across different types of datasets, we find that while
models trained on CAD show lower in-domain performance, they generalize better
out-of-domain. We unpack this apparent discrepancy using machine explanations
and find that CAD reduces model reliance on spurious features. Leveraging a
novel typology of CAD to analyze their relationship with model performance, we
find that CAD which acts on the construct directly or a diverse set of CAD
leads to higher performance.

研究如何提高自然语言处理模型的稳健性，发现使用对抗性生成数据可以降低模型在错误特征上的依赖，从而提高模型在领域外的泛化性能。