Two fundamental questions in neurolinguistics concerns the brain regions that
integrate information beyond the lexical level, and the size of their window of
integration. To address these questions we introduce a new approach named
masked-attention generation. It uses GPT-2 transformers to generate word
embeddings that capture a fixed amount of contextual information. We then
tested whether these embeddings could predict fMRI brain activity in humans
listening to naturalistic text. The results showed that most of the cortex
within the language network is sensitive to contextual information, and that
the right hemisphere is more sensitive to longer contexts than the left.
Masked-attention generation supports previous analyses of context-sensitivity
in the brain, and complements them by quantifying the window size of context
integration per voxel.

利用 GPT-2 变形金刚生成的单词嵌入来测试大脑对自然文本中背景信息的反应，结果表明语言网络中的大多数皮层对背景信息敏感，右半球对于较长场景的反应比左半球更敏感，支持以前的上下文敏感度分析并有助于量化每个体素的上下文整合窗口大小。