Releasing open-source large language models (LLMs) presents a dual-use risk
since bad actors can easily fine-tune these models for harmful purposes. Even
without the open release of weights, weight stealing and fine-tuning APIs make
closed models vulnerable to harmful fine-tuning attacks (HFAs). While safety
measures like preventing jailbreaks and improving safety guardrails are
important, such measures can easily be reversed through fine-tuning. In this
work, we propose Representation Noising (RepNoise), a defence mechanism that is
effective even when attackers have access to the weights and the defender no
longer has any control. RepNoise works by removing information about harmful
representations such that it is difficult to recover them during fine-tuning.
Importantly, our defence is also able to generalize across different subsets of
harm that have not been seen during the defence process. Our method does not
degrade the general capability of LLMs and retains the ability to train the
model on harmless tasks. We provide empirical evidence that the effectiveness
of our defence lies in its "depth": the degree to which information about
harmful representations is removed across all layers of the LLM.

我们提出了一种名为 Representation Noising (RepNoise) 的防御机制，它能在攻击者具有权重且防御者无法控制的情况下，有效地消除有害表示的信息，从而使恶意微调变得困难，并能在不同的有害子集上泛化，同时不降低大型语言模型的一般能力。