Knowledge editing for large language models can offer an efficient solution
to alter a model's behavior without negatively impacting the overall
performance. However, the current approach encounters issues with limited
generalizability across tasks, necessitating one distinct editor for each task,
which significantly hinders the broader applications. To address this, we take
the first step to analyze the multi-task generalization issue in knowledge
editing. Specifically, we develop an instruction-based editing technique,
termed InstructEdit, which facilitates the editor's adaptation to various task
performances simultaneously using simple instructions. With only one unified
editor for each LLM, we empirically demonstrate that InstructEdit can improve
the editor's control, leading to an average 14.86% increase in Reliability in
multi-task editing setting. Furthermore, experiments involving holdout unseen
task illustrate that InstructEdit consistently surpass previous strong
baselines. To further investigate the underlying mechanisms of
instruction-based knowledge editing, we analyze the principal components of the
editing gradient directions, which unveils that instructions can help control
optimization direction with stronger OOD generalization. Code and datasets will
be available in this https URL

大规模语言模型的知识编辑可以提供一种有效的解决方案，以改变模型的行为而不会对整体性能产生负面影响。为了解决当前方法在任务上的有限泛化能力的问题，我们采取了第一步来分析知识编辑中的多任务泛化问题。具体而言，我们开发了一种基于指令的编辑技术，称为 InstructEdit，它通过简单的指令同时促进编辑器对不同任务性能的适应。通过仅使用一个统一的编辑器为每个大规模语言模型，我们经验证明 InstructEdit 可以提高编辑器的控制能力，在多任务编辑设置中可平均提高 14.86％的可靠性。此外，涉及未见任务的实验表明 InstructEdit 始终优于先前的强基准。为了进一步研究基于指令的知识编辑的基本机制，我们分析了编辑梯度方向的主要成分，发现指令可以帮助控制具有更强的无法覆盖的泛化能力的优化方向。代码和数据集将在此 https URL 中提供。