As an efficient alternative to conventional full finetuning,
parameter-efficient finetuning (PEFT) is becoming the prevailing method to
adapt pretrained language models. In PEFT, a lightweight module is learned on
each dataset while the underlying pretrained language model remains unchanged,
resulting in multiple compact modules representing diverse skills when applied
to various domains and tasks. In this paper, we propose to compose these
parameter-efficient modules through linear arithmetic operations in the weight
space, thereby integrating different module capabilities. Specifically, we
first define addition and negation operators for the module, and then further
compose these two basic operators to perform flexible arithmetic. Our approach
requires \emph{no additional training} and enables highly flexible module
composition. We apply different arithmetic operations to compose the
parameter-efficient modules for (1) distribution generalization, (2)
multi-tasking, (3) unlearning, and (4) domain transfer. Additionally, we extend
our approach to detoxify Alpaca-LoRA, the latest instruction-tuned large
language model based on LLaMA. Empirical results demonstrate that our approach
produces new and effective parameter-efficient modules that significantly
outperform existing ones across all settings.

本文提出了一种通过线性算术运算在权重空间中组合不同模块能力的方法，可以用于参数有效的微调方法，可用于多任务、领域泛化和熟练度淡忘等。此方法可集成不同的模块能力，无需额外培训，并可用于 Alpaca-LoRA。实验结果表明，这种方法可以显着优于现有的方法。