Injecting textual information into knowledge graph (KG) entity
representations has been a worthwhile expedition in terms of improving
performance in KG oriented tasks within the NLP community. External knowledge
often adopted to enhance KG embeddings ranges from semantically rich lexical
dependency parsed features to a set of relevant key words to entire text
descriptions supplied from an external corpus such as wikipedia and many more.
Despite the gains this innovation (Text-enhanced KG embeddings) has made, the
proposal in this work suggests that it can be improved even further. Instead of
using a single text description (which would not sufficiently represent an
entity because of the inherent lexical ambiguity of text), we propose a
multi-task framework that jointly selects a set of text descriptions relevant
to KG entities as well as align or augment KG embeddings with text
descriptions. Different from prior work that plugs formal entity descriptions
declared in knowledge bases, this framework leverages a retriever model to
selectively identify richer or highly relevant text descriptions to use in
augmenting entities. Furthermore, the framework treats the number of
descriptions to use in augmentation process as a parameter, which allows the
flexibility of enumerating across several numbers before identifying an
appropriate number. Experiment results for Link Prediction demonstrate a 5.5%
and 3.5% percentage increase in the Mean Reciprocal Rank (MRR) and Hits@10
scores respectively, in comparison to text-enhanced knowledge graph
augmentation methods using traditional CNNs.

使用多任务框架，将文本描述与知识图谱实体相关联，以改进知识图谱的性能，并通过检索模型选择富有信息或高度相关的文本描述来增强实体。实验结果显示，与传统的卷积神经网络相比，使用文本增强的知识图谱方法在链接预测中的 Mean Reciprocal Rank (MRR) 和 Hits@10 分别提高了 5.5% 和 3.5%。

选择与增强：增强型密集检索知识图增强

Select and Augment: Enhanced Dense Retrieval Knowledge Graph  Augmentation

Recent approaches to Open-domain Question Answering refer to an external
knowledge base using a retriever model, optionally rerank passages with a
separate reranker model and generate an answer using another reader model.
Despite performing related tasks, the models have separate parameters and are
weakly-coupled during training. We propose casting the retriever and the
reranker as internal passage-wise attention mechanisms applied sequentially
within the transformer architecture and feeding computed representations to the
reader, with the hidden representations progressively refined at each stage.
This allows us to use a single question answering model trained end-to-end,
which is a more efficient use of model capacity and also leads to better
gradient flow. We present a pre-training method to effectively train this
architecture and evaluate our model on the Natural Questions and TriviaQA open
datasets. For a fixed parameter budget, our model outperforms the previous
state-of-the-art model by 1.0 and 0.7 exact match scores.

本研究提出了一种基于 transformer 架构的内部逐层注意力机制，将知识库搜索模型和重排模型整合为统一的模型，并进行端到端训练，以实现高效利用模型容量并提高梯度流量。该模型可以在固定参数预算内显著优于现有模型，达到 1.0 和 0.7 的精准匹配得分。