In this study, we propose an axiomatic system to define and quantify the
precise memorization and in-context reasoning effects used by the large
language model (LLM) for language generation. These effects are formulated as
non-linear interactions between tokens/words encoded by the LLM. Specifically,
the axiomatic system enables us to categorize the memorization effects into
foundational memorization effects and chaotic memorization effects, and further
classify in-context reasoning effects into enhanced inference patterns,
eliminated inference patterns, and reversed inference patterns. Besides, the
decomposed effects satisfy the sparsity property and the universal matching
property, which mathematically guarantee that the LLM's confidence score can be
faithfully decomposed into the memorization effects and in-context reasoning
effects. Experiments show that the clear disentanglement of memorization
effects and in-context reasoning effects enables a straightforward examination
of detailed inference patterns encoded by LLMs.

我们提出了一个公理系统来定义和量化大型语言模型（LLM）用于语言生成的精确记忆和上下文推理效应，并将这些效应形式化为 LLM 编码的令牌 / 词之间的非线性相互作用。实验表明，记忆效应和上下文推理效应的明确区分使得可以直观地检查 LLMs 编码的详细推理模式。

量化 LLM 中上下文推理效应和记忆效应

Quantifying In-Context Reasoning Effects and Memorization Effects in  LLMs

Recent studies on transformer-based language models show that they can answer
questions by reasoning over knowledge provided as part of the context (i.e.,
in-context reasoning). However, since the available knowledge is often not
filtered for a particular question, in-context reasoning can be sensitive to
distractor facts, additional content that is irrelevant to a question but that
may be relevant for a different question (i.e., not necessarily random noise).
In these situations, the model fails to distinguish the knowledge that is
necessary to answer the question, leading to spurious reasoning and degraded
performance. This reasoning failure contrasts with the model's apparent ability
to distinguish its contextual knowledge from all the knowledge it has memorized
during pre-training. Following this observation, we propose teaching the model
to reason more robustly by folding the provided contextual knowledge into the
model's parameters before presenting it with a question. Our method, RECKONING,
is a bi-level learning algorithm that teaches language models to reason by
updating their parametric knowledge through back-propagation, allowing them to
then answer questions using the updated parameters. During training, the inner
loop rapidly adapts a copy of the model weights to encode contextual knowledge
into its parameters. In the outer loop, the model learns to uses the updated
weights to reproduce and answer reasoning questions about the memorized
knowledge. Our experiments on two multi-hop reasoning datasets show that
RECKONING's performance improves over the in-context reasoning baseline (by up
to 4.5%). We also find that compared to in-context reasoning, RECKONING
generalizes better to longer reasoning chains unseen during training, is more
robust to distractors in the context, and is more computationally efficient
when multiple questions are asked about the same knowledge.

该研究提出了一种名为 RECKONING 的方法，通过双层学习算法将上下文知识整合到模型参数中，从而使语言模型更加强健，具有更好的推理能力和抗干扰性，从而提高了其多跳推理数据集的性能。