Knowledge embeddings (KE) represent a knowledge graph (KG) by embedding
entities and relations into continuous vector spaces. Existing methods are
mainly structure-based or description-based. Structure-based methods learn
representations that preserve the inherent structure of KGs. They cannot well
represent abundant long-tail entities in real-world KGs with limited structural
information. Description-based methods leverage textual information and
language models. Prior approaches in this direction barely outperform
structure-based ones, and suffer from problems like expensive negative sampling
and restrictive description demand. In this paper, we propose LMKE, which
adopts Language Models to derive Knowledge Embeddings, aiming at both enriching
representations of long-tail entities and solving problems of prior
description-based methods. We formulate description-based KE learning with a
contrastive learning framework to improve efficiency in training and
evaluation. Experimental results show that LMKE achieves state-of-the-art
performance on KE benchmarks of link prediction and triple classification,
especially for long-tail entities.

该论文提出了一种基于对比学习的 LMKE 方法，采用语言模型从文本信息中推导出知识嵌入，旨在丰富长尾实体的表征，并解决以前的基于描述的方法的问题，实验结果表明 LMKE 在知识嵌入基准测试中特别是长尾实体上取得了最先进的性能.

语言模型作为知识嵌入

Language Models as Knowledge Embeddings

Knowledge-aware methods have boosted a range of natural language processing
applications over the last decades. With the gathered momentum, knowledge
recently has been pumped into enormous attention in document summarization, one
of natural language processing applications. Previous works reported that
knowledge-embedded document summarizers excel at generating superior digests,
especially in terms of informativeness, coherence, and fact consistency. This
paper pursues to present the first systematic survey for the state-of-the-art
methodologies that embed knowledge into document summarizers. Particularly, we
propose novel taxonomies to recapitulate knowledge and knowledge embeddings
under the document summarization view. We further explore how embeddings are
generated in embedding learning architectures of document summarization models,
especially of deep learning models. At last, we discuss the challenges of this
topic and future directions.

本文是第一个系统调查将知识嵌入到文档摘要生成器中的最新方法的论文，现在知识已成为自然语言处理应用程序中的焦点，先前的工作表明，嵌入式知识文档摘要生成器在生成信息、连贯性和事实一致性方面表现卓越，我们进一步研究嵌入深度学习模型的嵌入学习体系结构如何生成嵌入的，最后讨论这个主题的挑战和未来方向。