Pretrained language models have been suggested as a possible alternative or
complement to structured knowledge bases. However, this emerging LM-as-KB
paradigm has so far only been considered in a very limited setting, which only
allows handling 21k entities whose single-token name is found in common LM
vocabularies. Furthermore, the main benefit of this paradigm, namely querying
the KB using a variety of natural language paraphrases, is underexplored so
far. Here, we formulate two basic requirements for treating LMs as KBs: (i) the
ability to store a large number facts involving a large number of entities and
(ii) the ability to query stored facts. We explore three entity representations
that allow LMs to represent millions of entities and present a detailed case
study on paraphrased querying of world knowledge in LMs, thereby providing a
proof-of-concept that language models can indeed serve as knowledge bases.

本研究探讨基于预训练语言模型作为知识库的 Paradigm，提出两个基本要求：存储大量涉及大量实体的事实和查询存储的事实的能力，并探索了三种实体表示法，提供了一个实证，证明语言模型确实可以作为知识库。