Language models are susceptible to bias, sycophancy, backdoors, and other
tendencies that lead to unfaithful responses to the input context. Interpreting
internal states of language models could help monitor and correct unfaithful
behavior. We hypothesize that language models represent their input contexts in
a latent world model, and seek to extract this latent world state from the
activations. We do so with 'propositional probes', which compositionally probe
tokens for lexical information and bind them into logical propositions
representing the world state. For example, given the input context ''Greg is a
nurse. Laura is a physicist.'', we decode the propositions ''WorksAs(Greg,
nurse)'' and ''WorksAs(Laura, physicist)'' from the model's activations. Key to
this is identifying a 'binding subspace' in which bound tokens have high
similarity (''Greg'' and ''nurse'') but unbound ones do not (''Greg'' and
''physicist''). We validate propositional probes in a closed-world setting with
finitely many predicates and properties. Despite being trained on simple
templated contexts, propositional probes generalize to contexts rewritten as
short stories and translated to Spanish. Moreover, we find that in three
settings where language models respond unfaithfully to the input context --
prompt injections, backdoor attacks, and gender bias -- the decoded
propositions remain faithful. This suggests that language models often encode a
faithful world model but decode it unfaithfully, which motivates the search for
better interpretability tools for monitoring LMs.

语言模型在其输入语境中容易受到偏见、阿谀奉承、后门等倾向的影响，解释语言模型的内部状态可以帮助监控和纠正不忠实的行为。本研究假设语言模型在一个潜在的世界模型中表示其输入语境，并通过 “命题探测器” 提取这个潜在的世界状态。