Large transformer-based pre-trained language models have achieved impressive
performance on a variety of knowledge-intensive tasks and can capture factual
knowledge in their parameters. We argue that storing large amounts of knowledge
in the model parameters is sub-optimal given the ever-growing amounts of
knowledge and resource requirements. We posit that a more efficient alternative
is to provide explicit access to contextually relevant structured knowledge to
the model and train it to use that knowledge. We present LM-CORE -- a general
framework to achieve this -- that allows \textit{decoupling} of the language
model training from the external knowledge source and allows the latter to be
updated without affecting the already trained model. Experimental results show
that LM-CORE, having access to external knowledge, achieves significant and
robust outperformance over state-of-the-art knowledge-enhanced language models
on knowledge probing tasks; can effectively handle knowledge updates; and
performs well on two downstream tasks. We also present a thorough error
analysis highlighting the successes and failures of LM-CORE.

论文讨论了使用外部结构化知识进行预处理以提升模型性能的方法，提出了 LM-CORE 框架作为一种更有效的替代方案，并通过实验结果表明具有外部结构化知识的 LM-CORE 在知识探究任务上具有显著的优越性，在两个下游任务的表现良好，并能有效处理知识更新问题。