A growing body of literature has focused on detailing the linguistic
knowledge embedded in large, pretrained language models. Existing work has
shown that non-linguistic biases in models can drive model behavior away from
linguistic generalizations. We hypothesized that competing linguistic processes
within a language, rather than just non-linguistic model biases, could obscure
underlying linguistic knowledge. We tested this claim by exploring a single
phenomenon in four languages: English, Chinese, Spanish, and Italian. While
human behavior has been found to be similar across languages, we find
cross-linguistic variation in model behavior. We show that competing processes
in a language act as constraints on model behavior and demonstrate that
targeted fine-tuning can re-weight the learned constraints, uncovering
otherwise dormant linguistic knowledge in models. Our results suggest that
models need to learn both the linguistic constraints in a language and their
relative ranking, with mismatches in either producing non-human-like behavior.

本文研究了大型预训练语言模型中内嵌的语言知识，并探索了同时存在的竞争性语言过程如何影响模型行为。通过对四种语言中的一个现象进行实验，发现模型行为存在跨语言差异。而有针对性的微调可以重新建立学习的约束，揭示模型中否则未表现出的语言知识。