Knowledge infusion is a promising method for enhancing Large Language Models
for domain-specific NLP tasks rather than pre-training models over large data
from scratch. These augmented LLMs typically depend on additional pre-training
or knowledge prompts from an existing knowledge graph, which is impractical in
many applications. In contrast, knowledge infusion directly from relevant
documents is more generalisable and alleviates the need for structured
knowledge graphs while also being useful for entities that are usually not
found in any knowledge graph. With this motivation, we propose a simple yet
generalisable approach for knowledge infusion by generating prompts from the
context in the input text. Our experiments show the effectiveness of our
approach which we evaluate by probing the fine-tuned LLMs.

提出了一种通过从输入文本中生成提示来实现知识注入的简单而通用的方法，从而增强大型语言模型在特定领域的自然语言处理任务中的性能，并通过对微调后的大型语言模型进行探测评估来证明该方法的有效性。

利用背景提示将知识注入大型语言模型

Infusing Knowledge into Large Language Models with Contextual Prompts

Pre-trained language models (PLMs) have demonstrated strong performance in
sequential recommendation (SR), which are utilized to extract general
knowledge. However, existing methods still lack domain knowledge and struggle
to capture users' fine-grained preferences. Meanwhile, many traditional SR
methods improve this issue by integrating side information while suffering from
information loss. To summarize, we believe that a good recommendation system
should utilize both general and domain knowledge simultaneously. Therefore, we
introduce an external knowledge base and propose Knowledge Prompt-tuning for
Sequential Recommendation (\textbf{KP4SR}). Specifically, we construct a set of
relationship templates and transform a structured knowledge graph (KG) into
knowledge prompts to solve the problem of the semantic gap. However, knowledge
prompts disrupt the original data structure and introduce a significant amount
of noise. We further construct a knowledge tree and propose a knowledge tree
mask, which restores the data structure in a mask matrix form, thus mitigating
the noise problem. We evaluate KP4SR on three real-world datasets, and
experimental results show that our approach outperforms state-of-the-art
methods on multiple evaluation metrics. Specifically, compared with PLM-based
methods, our method improves NDCG@5 and HR@5 by \textcolor{red}{40.65\%} and
\textcolor{red}{36.42\%} on the books dataset, \textcolor{red}{11.17\%} and
\textcolor{red}{11.47\%} on the music dataset, and \textcolor{red}{22.17\%} and
\textcolor{red}{19.14\%} on the movies dataset, respectively. Our code is
publicly available at the link:
\href{https://github.com/zhaijianyang/KP4SR}{\textcolor{blue}{this https URL}.}

我们提出了一种集成领域知识与通用知识的推荐系统，通过引入外部知识库构建关系模板和知识树，并利用知识提示和知识树掩码来解决语义差距和噪声问题，实验证明我们的方法在多个评估指标上优于现有方法。