Accounts of human language processing have long appealed to implicit
``situation models'' that enrich comprehension with relevant but unstated world
knowledge. Here, we apply causal intervention techniques to recent transformer
models to analyze performance on the Winograd Schema Challenge (WSC), where a
single context cue shifts interpretation of an ambiguous pronoun. We identify a
relatively small circuit of attention heads that are responsible for
propagating information from the context word that guides which of the
candidate noun phrases the pronoun ultimately attends to. We then compare how
this circuit behaves in a closely matched ``syntactic'' control where the
situation model is not strictly necessary. These analyses suggest distinct
pathways through which implicit situation models are constructed to guide
pronoun resolution.

通过对最近的变形器模型应用因果干预技术分析了词义消歧问题上 “情境模型” 的表现，特别是注意头回路在隐含情境模型建立中发挥的作用及其通过不同路径对代词解析的引导方式进行比较。

因果干预揭示了常识语言理解中的隐含情景模型

Causal interventions expose implicit situation models for commonsense  language understanding

The context in which a sentence appears can drastically alter our
expectations about upcoming words - for example, following a short story
involving an anthropomorphic peanut, experimental participants are more likely
to expect the sentence 'the peanut was in love' than 'the peanut was salted',
as indexed by N400 amplitude (Nieuwland & van Berkum, 2006). This rapid and
dynamic updating of comprehenders' expectations about the kind of events that a
peanut may take part in based on context has been explained using the construct
of Situation Models - updated mental representations of key elements of an
event under discussion, in this case, the peanut protagonist. However, recent
work showing that N400 amplitude can be predicted based on distributional
information alone raises the question whether situation models are in fact
necessary for the kinds of contextual effects observed in previous work. To
investigate this question, we attempt to model the results of Nieuwland and van
Berkum (2006) using six computational language models and three sets of word
vectors, none of which have explicit situation models or semantic grounding. We
find that the effect found by Nieuwland and van Berkum (2006) can be fully
modeled by two language models and two sets of word vectors, with others
showing a reduced effect. Thus, at least some processing effects normally
explained through situation models may not in fact require explicit situation
models.

通过计算语言模型和词向量，我们模拟了句子上下文对语义预期的影响，结果表明情境模型可能并不是解释上下文影响的唯一方式。