To tackle the problem of domain-specific knowledge scarcity within large
language models (LLMs), knowledge graph-retrievalaugmented method has been
proven to be an effective and efficient technique for knowledge infusion.
However, existing approaches face two primary challenges: knowledge mismatch
between public available knowledge graphs and the specific domain of the task
at hand, and poor information compliance of LLMs with knowledge graphs. In this
paper, we leverage a small set of labeled samples and a large-scale corpus to
efficiently construct domain-specific knowledge graphs by an LLM, addressing
the issue of knowledge mismatch. Additionally, we propose a three-stage KG-LLM
alignment strategyto enhance the LLM's capability to utilize information from
knowledge graphs. We conduct experiments with a limited-sample setting on two
biomedical question-answering datasets, and the results demonstrate that our
approach outperforms existing baselines.

针对大型语言模型（LLMs）中领域特定知识匮乏的问题，本文提出了一种基于知识图谱检索增强的方法，通过利用少量标记样本和大规模语料库，构建了领域特定的知识图，并通过三阶段的知识图与 LLM 对齐策略增强了 LLM 利用知识图的能力，实验证明我们的方法优于现有基准模型。

基于知识图谱的高效知识渗透

Efficient Knowledge Infusion via KG-LLM Alignment

Biomedical question-answering (QA) has gained increased attention for its
capability to provide users with high-quality information from a vast
scientific literature. Although an increasing number of biomedical QA datasets
has been recently made available, those resources are still rather limited and
expensive to produce. Transfer learning via pre-trained language models (LMs)
has been shown as a promising approach to leverage existing general-purpose
knowledge. However, finetuning these large models can be costly and time
consuming, often yielding limited benefits when adapting to specific themes of
specialised domains, such as the COVID-19 literature. To bootstrap further
their domain adaptation, we propose a simple yet unexplored approach, which we
call biomedical entity-aware masking (BEM). We encourage masked language models
to learn entity-centric knowledge based on the pivotal entities characterizing
the domain at hand, and employ those entities to drive the LM fine-tuning. The
resulting strategy is a downstream process applicable to a wide variety of
masked LMs, not requiring additional memory or components in the neural
architectures. Experimental results show performance on par with
state-of-the-art models on several biomedical QA datasets.

本文提出了一种称为生物医学实体感知蒙版 (BEM) 的简单方法，以帮助受限于特定领域及其 COVID-19 文献的预训练语言模型进行微调，从而提升其生物医学问答数据集的性能。实验证明，该方法与先进模型具有相同的性能。