Natural language inference (NLI) is an increasingly important task for
natural language understanding, which requires one to infer whether a sentence
entails another. However, the ability of NLI models to make pragmatic
inferences remains understudied. We create an IMPlicature and PRESupposition
diagnostic dataset (IMPPRES), consisting of >25k semiautomatically generated
sentence pairs illustrating well-studied pragmatic inference types. We use
IMPPRES to evaluate whether BERT, InferSent, and BOW NLI models trained on
MultiNLI (Williams et al., 2018) learn to make pragmatic inferences. Although
MultiNLI appears to contain very few pairs illustrating these inference types,
we find that BERT learns to draw pragmatic inferences. It reliably treats
scalar implicatures triggered by "some" as entailments. For some presupposition
triggers like "only", BERT reliably recognizes the presupposition as an
entailment, even when the trigger is embedded under an entailment canceling
operator like negation. BOW and InferSent show weaker evidence of pragmatic
reasoning. We conclude that NLI training encourages models to learn some, but
not all, pragmatic inferences.

本文在构建议程推理和预设的诊断数据集后，评估了在 MultiNLI 训练的 BERT、InferSent 和 BOW NLI 模型学习进行实用推理的能力。结果表明，尽管 MultiNLI 中的很少数的偶合类型表示，但 BERT 可以绘制实用偶合，而 BOW 和 InferSent 则表现出较弱的实用推理证据。

自然语言推理模型是否令人印象深刻？学习隐含和预设条件

Are Natural Language Inference Models IMPPRESsive? Learning IMPlicature  and PRESupposition

Modeling natural language inference is a very challenging task. With the
availability of large annotated data, it has recently become feasible to train
complex models such as neural-network-based inference models, which have shown
to achieve the state-of-the-art performance. Although there exist relatively
large annotated data, can machines learn all knowledge needed to perform
natural language inference (NLI) from these data? If not, how can
neural-network-based NLI models benefit from external knowledge and how to
build NLI models to leverage it? In this paper, we enrich the state-of-the-art
neural natural language inference models with external knowledge. We
demonstrate that the proposed models improve neural NLI models to achieve the
state-of-the-art performance on the SNLI and MultiNLI datasets.

本文研究在于如何用外部的知识来丰富神经网络的自然语言推理模型，并展示了这些模型如何在 SNLI 和 MultiNLI 数据集上实现最先进的性能水平。

带有外部知识的神经自然语言推理模型

Neural Natural Language Inference Models Enhanced with External  Knowledge

This paper presents the results of the RepEval 2017 Shared Task, which
evaluated neural network sentence representation learning models on the
Multi-Genre Natural Language Inference corpus (MultiNLI) recently introduced by
Williams et al. (2017). All of the five participating teams beat the
bidirectional LSTM (BiLSTM) and continuous bag of words baselines reported in
Williams et al.. The best single model used stacked BiLSTMs with residual
connections to extract sentence features and reached 74.5% accuracy on the
genre-matched test set. Surprisingly, the results of the competition were
fairly consistent across the genre-matched and genre-mismatched test sets, and
across subsets of the test data representing a variety of linguistic phenomena,
suggesting that all of the submitted systems learned reasonably
domain-independent representations for sentence meaning.

本文介绍了 RepEval 2017 共享任务的结果，该任务评估了神经网络句子表示学习模型在 MultiNLI 语料库上的表现。结果表明，所有五个参赛团队都击败了以前的标记方法。最佳单一模型使用堆叠 BiLSTM 和残差连接来提取句子特征，并在与体裁匹配的测试集上达到了 74.5% 的准确率。结果表明，提交的所有系统对于句子意义的领域独立表示都有着合理的学习能力。