Pre-trained language models (LMs) are used for knowledge intensive tasks like
question answering, but their knowledge gets continuously outdated as the world
changes. Prior work has studied targeted updates to LMs, injecting individual
facts and evaluating whether the model learns these facts while not changing
predictions on other contexts. We take a step forward and study LMs' abilities
to make inferences based on injected facts (or propagate those facts): for
example, after learning that something is a TV show, does an LM predict that
you can watch it? We study this with two cloze-style tasks: an existing dataset
of real-world sentences about novel entities (ECBD) as well as a new controlled
benchmark with manually designed templates requiring varying levels of
inference about injected knowledge. Surprisingly, we find that existing methods
for updating knowledge (gradient-based fine-tuning and modifications of this
approach) show little propagation of injected knowledge. These methods improve
performance on cloze instances only when there is lexical overlap between
injected facts and target inferences. Yet, prepending entity definitions in an
LM's context improves performance across all settings, suggesting that there is
substantial headroom for parameter-updating approaches for knowledge injection.

该研究探讨预训练语言模型进行知识注入和推理的能力，发现目前的更新方法存在学习效果局限性，但将实体定义放在上下文中可提高性能，表明参数更新方法的知识注入还有很大的潜力。