Model editing aims to correct outdated or erroneous knowledge in large
language models (LLMs) without the need for costly retraining. Lifelong model
editing is the most challenging task that caters to the continuous editing
requirements of LLMs. Prior works primarily focus on single or batch editing;
nevertheless, these methods fall short in lifelong editing scenarios due to
catastrophic knowledge forgetting and the degradation of model performance.
Although retrieval-based methods alleviate these issues, they are impeded by
slow and cumbersome processes of integrating the retrieved knowledge into the
model. In this work, we introduce RECIPE, a RetriEval-augmented ContInuous
Prompt lEarning method, to boost editing efficacy and inference efficiency in
lifelong learning. RECIPE first converts knowledge statements into short and
informative continuous prompts, prefixed to the LLM's input query embedding, to
efficiently refine the response grounded on the knowledge. It further
integrates the Knowledge Sentinel (KS) that acts as an intermediary to
calculate a dynamic threshold, determining whether the retrieval repository
contains relevant knowledge. Our retriever and prompt encoder are jointly
trained to achieve editing properties, i.e., reliability, generality, and
locality. In our experiments, RECIPE is assessed extensively across multiple
LLMs and editing datasets, where it achieves superior editing performance.
RECIPE also demonstrates its capability to maintain the overall performance of
LLMs alongside showcasing fast editing and inference speed.

RECIPE 是一种 RetriEval-augmented ContInuous Prompt Learning 方法，旨在提高模型编辑效果和推理效率，通过将知识语句转换为短且信息丰富的连续提示，并与知识哨兵（KS）结合使用，进一步集成检索器和提示编码器，实现在模型编辑中的可靠性、普适性和局部性。在多个大型语言模型和编辑数据集上的广泛实验中，RECIPE 取得了出色的编辑性能，并展示了快速编辑和推理速度，同时保持了大型语言模型的整体性能。