Large Language Models (LLMs) have significantly advanced natural language
processing (NLP) with their impressive language understanding and generation
capabilities. However, their performance may be suboptimal for long-tail or
domain-specific tasks due to limited exposure to domain-specific knowledge and
vocabulary. Additionally, the lack of transparency of most state-of-the-art
(SOTA) LLMs, which can only be accessed via APIs, impedes further fine-tuning
with custom data. Moreover, data privacy is a significant concern. To address
these challenges, we propose the novel Parametric Knowledge Guiding (PKG)
framework, which equips LLMs with a knowledge-guiding module to access relevant
knowledge at runtime without altering the LLMs' parameters. Our PKG is based on
open-source "white-box" small language models, allowing offline storage of any
knowledge that LLMs require. We demonstrate that our PKG framework can enhance
the performance of "black-box" LLMs on a range of long-tail and domain-specific
downstream tasks requiring factual, tabular, medical, and multimodal knowledge.

本论文提出了 Parametric Knowledge Guiding（PKG）框架，使 LLMs 具有在运行时访问相关知识的能力，从而在适应长尾或特定领域任务方面提高性能，并同时考虑了透明度和数据隐私的问题。