Pretrained language models (LMs) do not capture factual knowledge very well.
This has led to the development of a number of knowledge integration (KI)
methods which aim to incorporate external knowledge into pretrained LMs. Even
though KI methods show some performance gains over vanilla LMs, the
inner-workings of these methods are not well-understood. For instance, it is
unclear how and what kind of knowledge is effectively integrated into these
models and if such integration may lead to catastrophic forgetting of already
learned knowledge. This paper revisits the KI process in these models with an
information-theoretic view and shows that KI can be interpreted using a graph
convolution operation. We propose a probe model called \textit{Graph
Convolution Simulator} (GCS) for interpreting knowledge-enhanced LMs and
exposing what kind of knowledge is integrated into these models. We conduct
experiments to verify that our GCS can indeed be used to correctly interpret
the KI process, and we use it to analyze two well-known knowledge-enhanced LMs:
ERNIE and K-Adapter, and find that only a small amount of factual knowledge is
integrated in them. We stratify knowledge in terms of various relation types
and find that ERNIE and K-Adapter integrate different kinds of knowledge to
different extent. Our analysis also shows that simply increasing the size of
the KI corpus may not lead to better KI; fundamental advances may be needed.

该论文提出了一种叫做 “Graph Convolution Simulator”（GCS）的探针模型，用于解释知识增强语言模型（Knowledge-enhanced LMs）中知识整合的方式，并发现只有少量的事实知识被整合进 ERNIE 和 K-Adapter 中。