Relational knowledge bases (KBs) are commonly used to represent world
knowledge in machines. However, while advantageous for their high degree of
precision and interpretability, KBs are usually organized according to
manually-defined schemas, which limit their expressiveness and require
significant human efforts to engineer and maintain. In this review, we take a
natural language processing perspective to these limitations, examining how
they may be addressed in part by training deep contextual language models (LMs)
to internalize and express relational knowledge in more flexible forms. We
propose to organize knowledge representation strategies in LMs by the level of
KB supervision provided, from no KB supervision at all to entity- and
relation-level supervision. Our contributions are threefold: (1) We provide a
high-level, extensible taxonomy for knowledge representation in LMs; (2) Within
our taxonomy, we highlight notable models, evaluation tasks, and findings, in
order to provide an up-to-date review of current knowledge representation
capabilities in LMs; and (3) We suggest future research directions that build
upon the complementary aspects of LMs and KBs as knowledge representations.

该综述着重探讨如何通过训练深度上下文语言模型，在没有人为限制下更灵活地内部化和表达关系知识，对知识表示策略进行分类，提出了高层次的可扩展分类法，并强调了当前语言模型在知识表示方面的能力与未来的研究方向。