How is knowledge of position-role mappings in natural language learned? We
explore this question in a computational setting, testing whether a variety of
well-performing pertained language models (BERT, RoBERTa, and DistilBERT)
exhibit knowledge of these mappings, and whether this knowledge persists across
alternations in syntactic, structural, and lexical alternations. In Experiment
1, we show that these neural models do indeed recognize distinctions between
theme and recipient roles in ditransitive constructions, and that these
distinct patterns are shared across construction type. We strengthen this
finding in Experiment 2 by showing that fine-tuning these language models on
novel theme- and recipient-like tokens in one paradigm allows the models to
make correct predictions about their placement in other paradigms, suggesting
that the knowledge of these mappings is shared rather than independently
learned. We do, however, observe some limitations of this generalization when
tasks involve constructions with novel ditransitive verbs, hinting at a degree
of lexical specificity which underlies model performance.

本论文通过计算实验的方法研究了是否神经语言模型具有识别自然语言中位置 - 角色映射的知识，发现这些模型能够识别双及物式结构中的主题和接收者角色，并且这种识别能力具有跨系统的普适性，但也存在一定的词汇特异性的局限性。