Pre-trained language models (PLMs) cannot well recall rich factual knowledge
of entities exhibited in large-scale corpora, especially those rare entities.
In this paper, we propose to build a simple but effective Pluggable Entity
Lookup Table (PELT) on demand by aggregating the entity's output
representations of multiple occurrences in the corpora. PELT can be compatibly
plugged as inputs to infuse supplemental entity knowledge into PLMs. Compared
to previous knowledge-enhanced PLMs, PELT only requires 0.2%-5% pre-computation
with capability of acquiring knowledge from out-of-domain corpora for domain
adaptation scenario. The experiments on knowledge-related tasks demonstrate
that our method, PELT, can flexibly and effectively transfer entity knowledge
from related corpora into PLMs with different architectures.

本文提出了一种 PELE 方法，通过聚合出现在大规模语料库中的实体的输出表示，能够将相关语料库中的实体知识灵活有效地传输到具有不同架构的预训练语言模型中，只需要 0.2%-5% 的预计算能够在域自适应场景中从域外语料库获取知识，实验表明该方法比以前的知识增强预训练语言模型更为优秀。