RNN language models have achieved state-of-the-art results on various tasks,
but what exactly they are representing about syntax is as yet unclear. Here we
investigate whether RNN language models learn humanlike word order preferences
in syntactic alternations. We collect language model surprisal scores for
controlled sentence stimuli exhibiting major syntactic alternations in English:
heavy NP shift, particle shift, the dative alternation, and the genitive
alternation. We show that RNN language models reproduce human preferences in
these alternations based on NP length, animacy, and definiteness. We collect
human acceptability ratings for our stimuli, in the first acceptability
judgment experiment directly manipulating the predictors of syntactic
alternations. We show that the RNNs' performance is similar to the human
acceptability ratings and is not matched by an n-gram baseline model. Our
results show that RNNs learn the abstract features of weight, animacy, and
definiteness which underlie soft constraints on syntactic alternations.

通过构造符合主要句法变异的受控句子刺激，研究发现 RNN 语言模型能够复现人类在句法变异中基于名词短长、生命力和确定性等因素的偏好，并且能够与人类评估结果相匹配，考虑到 N-gram 基准模型的情况，这说明了 RNNs 学习了在句法变异中概括名词重量、生命力和确定性等抽象特征。