Recursive processing in sentence comprehension is considered a hallmark of
human linguistic abilities. However, its underlying neural mechanisms remain
largely unknown. We studied whether a modern artificial neural network trained
with "deep learning" methods mimics a central aspect of human sentence
processing, namely the storing of grammatical number and gender information in
working memory and its use in long-distance agreement (e.g., capturing the
correct number agreement between subject and verb when they are separated by
other phrases). Although the network, a recurrent architecture with Long
Short-Term Memory units, was solely trained to predict the next word in a large
corpus, analysis showed the emergence of a very sparse set of specialized units
that successfully handled local and long-distance syntactic agreement for
grammatical number. However, the simulations also showed that this mechanism
does not support full recursion and fails with some long-range embedded
dependencies. We tested the model's predictions in a behavioral experiment
where humans detected violations in number agreement in sentences with
systematic variations in the singular/plural status of multiple nouns, with or
without embedding. Human and model error patterns were remarkably similar,
showing that the model echoes various effects observed in human data. However,
a key difference was that, with embedded long-range dependencies, humans
remained above chance level, while the model's systematic errors brought it
below chance. Overall, our study shows that exploring the ways in which modern
artificial neural networks process sentences leads to precise and testable
hypotheses about human linguistic performance.

使用深度学习方法训练的人工神经网络可以在一定程度上模拟人类语句处理的神经机制，但由于某些长距离内嵌依赖的影响，尚不支持完全递归，并且在与人类检测单复数一致性方面具有类似的误差模式。