As neural language models approach human performance on NLP benchmark tasks,
their advances are widely seen as evidence of an increasingly complex
understanding of syntax. This view rests upon a hypothesis that has not yet
been empirically tested: that word order encodes meaning essential to
performing these tasks. We refute this hypothesis in many cases: in the GLUE
suite and in various genres of English text, the words in a sentence or phrase
can rarely be permuted to form a phrase carrying substantially different
information. Our surprising result relies on inference by iterative shuffling
(IBIS), a novel, efficient procedure that finds the ordering of a bag of words
having the highest likelihood under a fixed language model. IBIS can use any
black-box model without additional training and is superior to existing word
ordering algorithms. Coalescing our findings, we discuss how shuffling
inference procedures such as IBIS can benefit language modeling and constrained
generation.

通过使用一种新的称为 IBIS 的过程，本文否认了一种假说，即单词顺序对于进行 NLP 基准任务而言必须编码含义的观点，我们在 GLUE 套件和各种英语文本流派中的多种情况下证明这一点，并探讨了诸如 IBIS 这样的洗牌推理过程如何有益于语言建模和约束生成。