Languages typically provide more than one grammatical construction to express
certain types of messages. A speaker's choice of construction is known to
depend on multiple factors, including the choice of main verb -- a phenomenon
known as \emph{verb bias}. Here we introduce DAIS, a large benchmark dataset
containing 50K human judgments for 5K distinct sentence pairs in the English
dative alternation. This dataset includes 200 unique verbs and systematically
varies the definiteness and length of arguments. We use this dataset, as well
as an existing corpus of naturally occurring data, to evaluate how well recent
neural language models capture human preferences. Results show that larger
models perform better than smaller models, and transformer architectures (e.g.
GPT-2) tend to out-perform recurrent architectures (e.g. LSTMs) even under
comparable parameter and training settings. Additional analyses of internal
feature representations suggest that transformers may better integrate specific
lexical information with grammatical constructions.

通过大规模数据集 DAIS，本文研究谓语偏向性对人类习惯表述的影响，探究神经语言模型对人类偏好的表达能力，结果表明基于 transformer 架构较基于循环神经网络较佳，并具有更好的语法信息融合表达能力。