This paper studies the problem of injecting factual knowledge into large
pre-trained language models. We train adapter modules on parts of the
ConceptNet knowledge graph using the masked language modeling objective and
evaluate the success of the method by a series of probing experiments on the
LAMA probe. Mean P@K curves for different configurations indicate that the
technique is effective, increasing the performance on subsets of the LAMA probe
for large values of k by adding as little as 2.1% additional parameters to the
original models.

本文研究了在大型预训练语言模型中注入实际知识的问题，并使用遮蔽语言建模目标训练 adapter 模块来评估该方法的成功性，证明该技术是有效的，将额外增加不到 2.1% 的参数到原始模型中即可提高在 LAMA 探针子集上的性能。

掩码语言建模和适配器对事实知识注入的有效性

The Effectiveness of Masked Language Modeling and Adapters for Factual Knowledge Injection

Cross-domain sentiment analysis has received significant attention in recent
years, prompted by the need to combat the domain gap between different
applications that make use of sentiment analysis. In this paper, we take a
novel perspective on this task by exploring the role of external commonsense
knowledge. We introduce a new framework, KinGDOM, which utilizes the ConceptNet
knowledge graph to enrich the semantics of a document by providing both
domain-specific and domain-general background concepts. These concepts are
learned by training a graph convolutional autoencoder that leverages
inter-domain concepts in a domain-invariant manner. Conditioning a popular
domain-adversarial baseline method with these learned concepts helps improve
its performance over state-of-the-art approaches, demonstrating the efficacy of
our proposed framework.

本文提出了一种新的框架 KinGDOM，利用 ConceptNet 知识图谱中的通用知识和领域特定知识来丰富文档的语义，通过训练图卷积自编码器来学习交叉领域知识，同时利用外部常识知识来改进领域敌对基准方法的性能。它在交叉领域情感分析任务中展现出比其他最先进的方法更好的性能。