Recently, Locate-Then-Edit paradigm has emerged as one of the main approaches
in changing factual knowledge stored in the Language models. However, there is
a lack of research on whether present locating methods can pinpoint the exact
parameters embedding the desired knowledge. Moreover, although many researchers
have questioned the validity of locality hypothesis of factual knowledge, no
method is provided to test the a hypothesis for more in-depth discussion and
research. Therefore, we introduce KLoB, a benchmark examining three essential
properties that a reliable knowledge locating method should satisfy. KLoB can
serve as a benchmark for evaluating existing locating methods in language
models, and can contributes a method to reassessing the validity of locality
hypothesis of factual knowledge. Our is publicly available at
https://github.com/juyiming/KLoB.

近期，Locate-Then-Edit 范式已成为改变语言模型中存储的事实知识的主要方法之一，然而，对于现有的定位方法是否能够准确定位到所需的知识嵌入的确切参数缺乏研究，此外，虽然许多研究人员对事实知识的局部性假设的有效性提出了质疑，但并未提供测试该假设以进行更深入讨论和研究的方法。因此，我们介绍了 KLoB，这是一个评估可靠的语言模型中的现有定位方法的三个基本属性的基准，KLoB 可以作为一个基准来评估语言模型中现有的定位方法，并提供了一种重新评估事实知识的局部性假设的方法。我们的方法公开在 https://github.com/juyiming/KLoB。