Structural locality is a ubiquitous feature of real-world datasets, wherein
data points are organized into local hierarchies. Some examples include topical
clusters in text or project hierarchies in source code repositories. In this
paper, we explore utilizing this structural locality within non-parametric
language models, which generate sequences that reference retrieved examples
from an external source. We propose a simple yet effective approach for adding
locality information into such models by adding learned parameters that improve
the likelihood of retrieving examples from local neighborhoods. Experiments on
two different domains, Java source code and Wikipedia text, demonstrate that
locality features improve model efficacy over models without access to these
features, with interesting differences. We also perform an analysis of how and
where locality features contribute to improved performance and why the
traditionally used contextual similarity metrics alone are not enough to grasp
the locality structure.

本文探讨了利用结构局部性在非参数语言模型中的应用，并提出了一种简单而有效的方法来将学习参数加入到这些模型中以提高局部邻域中检索示例的概率，并在 Java 源代码和维基百科文本等不同领域进行了实验，发现结构局部性特征可以改善模型效果。