The location of knowledge within Generative Pre-trained Transformer
(GPT)-like models has seen extensive recent investigation. However, much of the
work is focused towards determining locations of individual facts, with the end
goal being the editing of facts that are outdated, erroneous, or otherwise
harmful, without the time and expense of retraining the entire model. In this
work, we investigate a broader view of knowledge location, that of concepts or
clusters of related information, instead of disparate individual facts. To do
this, we first curate a novel dataset, called DARC, that includes a total of 34
concepts of ~120K factual statements divided into two types of hierarchical
categories, namely taxonomy and meronomy. Next, we utilize existing causal
mediation analysis methods developed for determining regions of importance for
individual facts and apply them to a series of related categories to provide
detailed investigation into whether concepts are associated with distinct
regions within these models. We find that related categories exhibit similar
areas of importance in contrast to less similar categories. However,
fine-grained localization of individual category subsets to specific regions is
not apparent.

在这项工作中，我们研究了知识定位的更广泛视角，即相关信息的概念或聚类，而不是孤立的个别事实。通过利用现有的因果中介分析方法，我们发现相关类别在模型中呈现类似的重要性区域，但无法明确特定区域内的个别类别子集的细粒度定位。

超越个体事实：GPT 模型中分类知识局部性的调查研究

Beyond Individual Facts: Investigating Categorical Knowledge Locality of  Taxonomy and Meronomy Concepts in GPT Models

Large Language Models (LLMs) have shown extraordinary capabilities in
understanding and generating text that closely mirrors human communication.
However, a primary limitation lies in the significant computational demands
during training, arising from their extensive parameterization. This challenge
is further intensified by the dynamic nature of the world, necessitating
frequent updates to LLMs to correct outdated information or integrate new
knowledge, thereby ensuring their continued relevance. Note that many
applications demand continual model adjustments post-training to address
deficiencies or undesirable behaviors. There is an increasing interest in
efficient, lightweight methods for on-the-fly model modifications. To this end,
recent years have seen a burgeoning in the techniques of knowledge editing for
LLMs, which aim to efficiently modify LLMs' behaviors within specific domains
while preserving overall performance across various inputs. In this paper, we
first define the knowledge editing problem and then provide a comprehensive
review of cutting-edge approaches. Drawing inspiration from educational and
cognitive research theories, we propose a unified categorization criterion that
classifies knowledge editing methods into three groups: resorting to external
knowledge, merging knowledge into the model, and editing intrinsic knowledge.
Furthermore, we introduce a new benchmark, KnowEdit, for a comprehensive
empirical evaluation of representative knowledge editing approaches.
Additionally, we provide an in-depth analysis of knowledge location, which can
provide a deeper understanding of the knowledge structures inherent within
LLMs. Finally, we discuss several potential applications of knowledge editing,
outlining its broad and impactful implications.

大型语言模型 (Large Language Models) 在理解和生成与人类交流非常相似的文本方面表现出非凡的能力。然而，由于其广泛的参数化，训练过程中存在重大的计算需求限制。这种挑战由于世界的动态性而进一步加剧，需要经常更新 LLM 以纠正过时的信息或整合新知识，从而确保它们持续具有相关性。近年来，对于即时修改模型的高效轻量级方法引起了越来越多的关注。本文首先定义了知识编辑问题，然后提供了对最前沿方法的全面回顾。在教育和认知研究理论的启发下，我们提出了一个统一的分类准则，将知识编辑方法分为三类：利用外部知识、将知识合并到模型中以及编辑内在知识。此外，我们引入了一个新的基准 KnowEdit，以对代表性的知识编辑方法进行综合实证评估。此外，我们对知识定位进行了深入分析，从而更深入地了解 LLM 固有的知识结构。最后，我们讨论了知识编辑的几个潜在应用，并概述了它的广泛而有影响力的意义。