Though Large Language Models (LLMs) have shown remarkable open-generation
capabilities across diverse domains, they struggle with knowledge-intensive
tasks. To alleviate this issue, knowledge integration methods have been
proposed to enhance LLMs with domain-specific knowledge graphs using external
modules. However, they suffer from data inefficiency as they require both known
and unknown knowledge for fine-tuning. Thus, we study a novel problem of
integrating unknown knowledge into LLMs efficiently without unnecessary overlap
of known knowledge. Injecting new knowledge poses the risk of forgetting
previously acquired knowledge. To tackle this, we propose a novel
Infuser-Guided Knowledge Integration (InfuserKI) framework that utilizes
transformer internal states to determine whether to enhance the original LLM
output with additional information, thereby effectively mitigating knowledge
forgetting. Evaluations on the UMLS-2.5k and MetaQA domain knowledge graphs
demonstrate that InfuserKI can effectively acquire new knowledge and outperform
state-of-the-art baselines by 9% and 6%, respectively, in reducing knowledge
forgetting.

通过使用 Infuser-Guided Knowledge Integration (InfuserKI) 框架，我们能够高效地将未知的知识集成到 Large Language Models (LLMs) 中，从而有效地减少知识遗忘，并超过现有方法在减少知识遗忘上的表现。