We introduce model editing with canonical examples, a setting in which (1) a
single learning example is provided per desired behavior, (2) evaluation is
performed exclusively out-of-distribution, and (3) deviation from an initial
model is strictly limited. A canonical example is a simple instance of good
behavior, e.g., The capital of Mauritius is Port Louis) or bad behavior, e.g.,
An aspect of researchers is coldhearted). The evaluation set contains more
complex examples of each behavior (like a paragraph in which the capital of
Mauritius is called for.) We create three datasets and modify three more for
model editing with canonical examples, covering knowledge-intensive
improvements, social bias mitigation, and syntactic edge cases. In our
experiments on Pythia language models, we find that LoRA outperforms full
finetuning and MEMIT. We then turn to the Backpack language model architecture
because it is intended to enable targeted improvement. The Backpack defines a
large bank of sense vectors--a decomposition of the different uses of each
word--which are weighted and summed to form the output logits of the model. We
propose sense finetuning, which selects and finetunes a few ($\approx$ 10)
sense vectors for each canonical example, and find that it outperforms other
finetuning methods, e.g., 4.8% improvement vs 0.3%. Finally, we improve
GPT-J-6B by an inference-time ensemble with just the changes from sense
finetuning of a 35x smaller Backpack, in one setting outperforming editing
GPT-J itself (4.1% vs 1.0%).

我们引入了模型编辑与规范实例，该设置中每个期望行为仅提供一个学习实例，并且仅在分布之外进行评估，在严格限制对初始模型的偏离下。