Deep Language Models (DLMs) provide a novel computational paradigm for
understanding the mechanisms of natural language processing in the human brain.
Unlike traditional psycholinguistic models, DLMs use layered sequences of
continuous numerical vectors to represent words and context, allowing a
plethora of emerging applications such as human-like text generation. In this
paper we show evidence that the layered hierarchy of DLMs may be used to model
the temporal dynamics of language comprehension in the brain by demonstrating a
strong correlation between DLM layer depth and the time at which layers are
most predictive of the human brain. Our ability to temporally resolve
individual layers benefits from our use of electrocorticography (ECoG) data,
which has a much higher temporal resolution than noninvasive methods like fMRI.
Using ECoG, we record neural activity from participants listening to a
30-minute narrative while also feeding the same narrative to a high-performing
DLM (GPT2-XL). We then extract contextual embeddings from the different layers
of the DLM and use linear encoding models to predict neural activity. We first
focus on the Inferior Frontal Gyrus (IFG, or Broca's area) and then extend our
model to track the increasing temporal receptive window along the linguistic
processing hierarchy from auditory to syntactic and semantic areas. Our results
reveal a connection between human language processing and DLMs, with the DLM's
layer-by-layer accumulation of contextual information mirroring the timing of
neural activity in high-order language areas.

深度语言模型（DLMs）为理解人脑中自然语言处理机制提供了一种新的计算范式。通过使用分层连续数值向量来表示单词和上下文，DLMs 与传统的心理语言学模型不同，从而产生了大量新兴应用，如人类般的文本生成。本文通过展示 DLMs 的分层层次结构可以模拟大脑中语言理解的时间动态性，从而证明了 DLMs 的层深与层是否最能预测人脑之间存在强相关。我们利用高时序分辨率的电皮层图谱（ECoG）数据记录参与者在听取一个 30 分钟叙述时的神经活动，同时将相同的叙述输入高性能 DLM（GPT2-XL）中。然后，我们提取 DLM 的不同层中的上下文嵌入，并使用线性编码模型预测神经活动。首先，我们关注下额下回脑回（IFG），然后扩展我们的模型，跟踪语言处理层次结构中从听觉到句法和语义区域的渐增时态感受窗口。我们的结果揭示了人类语言处理和 DLMs 之间的联系，DLM 的信息逐层累积上下文与高级语言区域的神经活动时机形成镜像关系。