Recent investigations into the inner-workings of state-of-the-art large-scale
pre-trained Transformer-based Natural Language Understanding (NLU) models
indicate that they appear to know humanlike syntax, at least to some extent. We
provide novel evidence that complicates this claim: we find that
state-of-the-art Natural Language Inference (NLI) models assign the same labels
to permuted examples as they do to the original, i.e. they are largely
invariant to random word-order permutations. This behavior notably differs from
that of humans; we struggle with ungrammatical sentences. To measure the
severity of this issue, we propose a suite of metrics and investigate which
properties of particular permutations lead models to be word-order invariant.
In the MNLI dataset, for example, we find almost all (98.7%) examples contain
at least one permutation which elicits the gold label. Models are sometimes
even able to assign gold labels to permutations that they originally failed to
predict correctly. We provide a comprehensive empirical evaluation of this
phenomenon, and further show that this issue exists for both Transformers and
pre-Transformer RNN / ConvNet based encoders, as well as across multiple
languages (English and Mandarin Chinese). Our code and data are available at
this https URL

通过一个明确的指标，我们提出了多种度量方法，并研究了哪些特定的排列方式使得大型 Transformer NLU 模型在自然语言推理任务中对随机词序具有一定程度的规律性。我们对此现象进行了综合实证评估，发现无论是基于 Transformer 还是基于 pre-Transformer RNN / ConvNet 结构的编码器，以及跨多种语言（英语和中文）都存在此问题，并提供相关数据和代码。

非自然语言推论

UnNatural Language Inference

If the same neural network architecture is trained multiple times on the same
dataset, will it make similar linguistic generalizations across runs? To study
this question, we fine-tuned 100 instances of BERT on the Multi-genre Natural
Language Inference (MNLI) dataset and evaluated them on the HANS dataset, which
evaluates syntactic generalization in natural language inference. On the MNLI
development set, the behavior of all instances was remarkably consistent, with
accuracy ranging between 83.6% and 84.8%. In stark contrast, the same models
varied widely in their generalization performance. For example, on the simple
case of subject-object swap (e.g., determining that "the doctor visited the
lawyer" does not entail "the lawyer visited the doctor"), accuracy ranged from
0.00% to 66.2%. Such variation is likely due to the presence of many local
minima that are equally attractive to a low-bias learner such as a neural
network; decreasing the variability may therefore require models with stronger
inductive biases.

本研究研究了同一神经网络架构在同一数据集上多次训练后是否在不同运行中进行相似的语言概括，通过在 Multi-genre 自然语言推理（MNLI）数据集上微调 100 个 BERT 实例并在评估自然语言推理中的句法概括的 HANS 数据集上对它们进行了评估，这类变异很可能是由于在局部极小值处的神经网络中出现的损失函数梯度等的不同选择而存在的，而减少变异可能需要具有更强归纳偏差的模型。