Phenomenon-specific "adversarial" datasets have been recently designed to
perform targeted stress-tests for particular inference types. Recent work (Liu
et al., 2019a) proposed that such datasets can be utilized for training NLI and
other types of models, often allowing to learn the phenomenon in focus and
improve on the challenge dataset, indicating a "blind spot" in the original
training data. Yet, although a model can improve in such a training process, it
might still be vulnerable to other challenge datasets targeting the same
phenomenon but drawn from a different distribution, such as having a different
syntactic complexity level. In this work, we extend this method to drive
conclusions about a model's ability to learn and generalize a target phenomenon
rather than to "learn" a dataset, by controlling additional aspects in the
adversarial datasets. We demonstrate our approach on two inference phenomena -
dative alternation and numerical reasoning, elaborating, and in some cases
contradicting, the results of Liu et al.. Our methodology enables building
better challenge datasets for creating more robust models, and may yield better
model understanding and subsequent overarching improvements.

本文提出将现象特定的 “对抗” 数据集用于训练 NLI 和其他类型的模型，表明了原始训练数据中的 “盲区”，并演示了用两个推理现象对该方法进行扩展来推动模型学习和推广目标现象的能力，从而建立更好的挑战数据集以创建更强大且更具鲁棒性的模型。