Abstract grammatical knowledge - of parts of speech and grammatical patterns
- is key to the capacity for linguistic generalization in humans. But how
abstract is grammatical knowledge in large language models? In the human
literature, compelling evidence for grammatical abstraction comes from
structural priming. A sentence that shares the same grammatical structure as a
preceding sentence is processed and produced more readily. Because confounds
exist when using stimuli in a single language, evidence of abstraction is even
more compelling from crosslingual structural priming, where use of a syntactic
structure in one language primes an analogous structure in another language. We
measure crosslingual structural priming in large language models, comparing
model behavior to human experimental results from eight crosslingual
experiments covering six languages, and four monolingual structural priming
experiments in three non-English languages. We find evidence for abstract
monolingual and crosslingual grammatical representations in the models that
function similarly to those found in humans. These results demonstrate that
grammatical representations in multilingual language models are not only
similar across languages, but they can causally influence text produced in
different languages.

大型语言模型中存在类似人类的抽象的单语和跨语言语法表示，这些语法表示在不同语言中可以产生因果影响。