While deep learning models are making fast progress on the task of Natural
Language Inference, recent studies have also shown that these models achieve
high accuracy by exploiting several dataset biases, and without deep
understanding of the language semantics. Using contradiction-word bias and
word-overlapping bias as our two bias examples, this paper explores both
data-level and model-level debiasing methods to robustify models against
lexical dataset biases. First, we debias the dataset through data augmentation
and enhancement, but show that the model bias cannot be fully removed via this
method. Next, we also compare two ways of directly debiasing the model without
knowing what the dataset biases are in advance. The first approach aims to
remove the label bias at the embedding level. The second approach employs a
bag-of-words sub-model to capture the features that are likely to exploit the
bias and prevents the original model from learning these biased features by
forcing orthogonality between these two sub-models. We performed evaluations on
new balanced datasets extracted from the original MNLI dataset as well as the
NLI stress tests, and show that the orthogonality approach is better at
debiasing the model while maintaining competitive overall accuracy. Our code
and data are available at: this https URL

本文探讨了深度学习模型在自然语言推理方面存在数据偏见的现象，并提出了数据级和模型级去偏见的方法，实验表明采用正交性方法可以更好地去偏见且保持高准确率。