Do state-of-the-art models for language understanding already have, or can
they easily learn, abilities such as boolean coordination, quantification,
conditionals, comparatives, and monotonicity reasoning (i.e., reasoning about
word substitutions in sentential contexts)? While such phenomena are involved
in natural language inference (NLI) and go beyond basic linguistic
understanding, it is unclear the extent to which they are captured in existing
NLI benchmarks and effectively learned by models. To investigate this, we
propose the use of semantic fragments---systematically generated datasets that
each target a different semantic phenomenon---for probing, and efficiently
improving, such capabilities of linguistic models. This approach to creating
challenge datasets allows direct control over the semantic diversity and
complexity of the targeted linguistic phenomena, and results in a more precise
characterization of a model's linguistic behavior. Our experiments, using a
library of 8 such semantic fragments, reveal two remarkable findings: (a)
State-of-the-art models, including BERT, that are pre-trained on existing NLI
benchmark datasets perform poorly on these new fragments, even though the
phenomena probed here are central to the NLI task. (b) On the other hand, with
only a few minutes of additional fine-tuning---with a carefully selected
learning rate and a novel variation of "inoculation"---a BERT-based model can
master all of these logic and monotonicity fragments while retaining its
performance on established NLI benchmarks.

通过使用语义片段，作者发现目前自然语言理解领域最先进的模型在布尔协调、量化、条件、比较和单调性推理等方面表现不佳，在这些问题上进行微调，BERT-based 模型可以掌握所有逻辑和单调性片段，同时保持其在已有 NLI 基准测试的性能。

通过语义片段探究自然语言推理模型

Probing Natural Language Inference Models through Semantic Fragments

In this paper, we propose multimodal convolutional neural networks (m-CNNs)
for matching image and sentence. Our m-CNN provides an end-to-end framework
with convolutional architectures to exploit image representation, word
composition, and the matching relations between the two modalities. More
specifically, it consists of one image CNN encoding the image content, and one
matching CNN learning the joint representation of image and sentence. The
matching CNN composes words to different semantic fragments and learns the
inter-modal relations between image and the composed fragments at different
levels, thus fully exploit the matching relations between image and sentence.
Experimental results on benchmark databases of bidirectional image and sentence
retrieval demonstrate that the proposed m-CNNs can effectively capture the
information necessary for image and sentence matching. Specifically, our
proposed m-CNNs for bidirectional image and sentence retrieval on Flickr30K and
Microsoft COCO databases achieve the state-of-the-art performances.

本论文提出了多模态卷积神经网络 (m-CNNs)，用于匹配图像和句子。该网络结构采用卷积架构来利用图像表示、单词组合和两种模态之间的匹配关系。实验结果表明，我们的 m-CNNs 可以有效地捕捉图像和句子匹配所需的信息，并在 Flickr30K 和 Microsoft COCO 数据库的双向图像和句子检索上取得了最先进的性能。