Knowledge editing for large language models can offer an efficient solution
to alter a model's behavior without negatively impacting the overall
performance. However, the current approach encounters issues with limited
generalizability across tasks, necessitating one distinct editor for each task,
which significantly hinders the broader applications. To address this, we take
the first step to analyze the multi-task generalization issue in knowledge
editing. Specifically, we develop an instruction-based editing technique,
termed InstructEdit, which facilitates the editor's adaptation to various task
performances simultaneously using simple instructions. With only one unified
editor for each LLM, we empirically demonstrate that InstructEdit can improve
the editor's control, leading to an average 14.86% increase in Reliability in
multi-task editing setting. Furthermore, experiments involving holdout unseen
task illustrate that InstructEdit consistently surpass previous strong
baselines. To further investigate the underlying mechanisms of
instruction-based knowledge editing, we analyze the principal components of the
editing gradient directions, which unveils that instructions can help control
optimization direction with stronger OOD generalization. Code and datasets will
be available in this https URL

大规模语言模型的知识编辑可以提供一种有效的解决方案，以改变模型的行为而不会对整体性能产生负面影响。为了解决当前方法在任务上的有限泛化能力的问题，我们采取了第一步来分析知识编辑中的多任务泛化问题。具体而言，我们开发了一种基于指令的编辑技术，称为 InstructEdit，它通过简单的指令同时促进编辑器对不同任务性能的适应。通过仅使用一个统一的编辑器为每个大规模语言模型，我们经验证明 InstructEdit 可以提高编辑器的控制能力，在多任务编辑设置中可平均提高 14.86％的可靠性。此外，涉及未见任务的实验表明 InstructEdit 始终优于先前的强基准。为了进一步研究基于指令的知识编辑的基本机制，我们分析了编辑梯度方向的主要成分，发现指令可以帮助控制具有更强的无法覆盖的泛化能力的优化方向。代码和数据集将在此 https URL 中提供。

InstructEdit：用于大型语言模型的指令基础知识编辑

InstructEdit: Instruction-based Knowledge Editing for Large Language  Models

Recent works have explored text-guided image editing using diffusion models
and generated edited images based on text prompts. However, the models struggle
to accurately locate the regions to be edited and faithfully perform precise
edits. In this work, we propose a framework termed InstructEdit that can do
fine-grained editing based on user instructions. Our proposed framework has
three components: language processor, segmenter, and image editor. The first
component, the language processor, processes the user instruction using a large
language model. The goal of this processing is to parse the user instruction
and output prompts for the segmenter and captions for the image editor. We
adopt ChatGPT and optionally BLIP2 for this step. The second component, the
segmenter, uses the segmentation prompt provided by the language processor. We
employ a state-of-the-art segmentation framework Grounded Segment Anything to
automatically generate a high-quality mask based on the segmentation prompt.
The third component, the image editor, uses the captions from the language
processor and the masks from the segmenter to compute the edited image. We
adopt Stable Diffusion and the mask-guided generation from DiffEdit for this
purpose. Experiments show that our method outperforms previous editing methods
in fine-grained editing applications where the input image contains a complex
object or multiple objects. We improve the mask quality over DiffEdit and thus
improve the quality of edited images. We also show that our framework can
accept multiple forms of user instructions as input. We provide the code at
this https URL

本文提出了一种名为 InstructEdit 的框架，其中包括了语言处理器，分段器和图像编辑器三个组件，这种框架可以根据用户的指令进行细粒度的编辑，且在输入图像中包含复杂对象或多个对象的精细编辑应用中表现优异。