Parameter-efficient finetuning (PEFT) is a key technique for adapting large
language models (LLMs) to downstream tasks. In this paper, we study leveraging
knowledge graph embeddings to improve the effectiveness of PEFT. We propose a
knowledgeable adaptation method called KnowLA. It inserts an adaptation layer
into an LLM to integrate the embeddings of entities appearing in the input
text. The adaptation layer is trained in combination with LoRA on instruction
data. Experiments on six benchmarks with two popular LLMs and three knowledge
graphs demonstrate the effectiveness and robustness of KnowLA. We show that
\modelname can help activate the relevant parameterized knowledge in an LLM to
answer a question without changing its parameters or input prompts.

通过将知识图嵌入到大型语言模型中，我们提出了一种名为 KnowLA 的知识适应方法，该方法通过在语言模型中插入适应层，将文本中出现的实体的嵌入融入到语言模型中，并与 LoRA 一起进行训练。实验结果表明 KnowLA 的有效性和鲁棒性，能够在不改变语言模型参数或输入提示的情况下，激活语言模型中的相关参数化知识以回答问题。