We address whether neural models for Natural Language Inference (NLI) can
learn the compositional interactions between lexical entailment and negation,
using four methods: the behavioral evaluation methods of (1) challenge test
sets and (2) systematic generalization tasks, and the structural evaluation
methods of (3) probes and (4) interventions. To facilitate this holistic
evaluation, we present Monotonicity NLI (MoNLI), a new naturalistic dataset
focused on lexical entailment and negation. In our behavioral evaluations, we
find that models trained on general-purpose NLI datasets fail systematically on
MoNLI examples containing negation, but that MoNLI fine-tuning addresses this
failure. In our structural evaluations, we look for evidence that our
top-performing BERT-based model has learned to implement the monotonicity
algorithm behind MoNLI. Probes yield evidence consistent with this conclusion,
and our intervention experiments bolster this, showing that the causal dynamics
of the model mirror the causal dynamics of this algorithm on subsets of MoNLI.
This suggests that the BERT model at least partially embeds a theory of lexical
entailment and negation at an algorithmic level.

本研究通过四种方法评估自然语言推理 (NLI) 模型是否可以学习词汇蕴涵和否定之间的组合交互作用，并提出了一个新的自然数据集 MoNLI，其中集中了词汇蕴涵和否定的相关内容，通过对 MoNLI 的 Fine-tuning 得到的模型相较于通用 NLI 数据集拥有更好的效果，同时对当前表现最佳的 BERT 模型进行探究表明其至少部分嵌入了词汇蕴涵和否定的算法级别理论。