The new paradigm of finetuning-as-a-service introduces a new attack surface
for Large Language Models (LLMs): a few harmful data uploaded by users can
easily trick the finetuning to produce an alignment-broken model. We conduct an
empirical analysis and uncover a \textit{harmful embedding drift} phenomenon,
showing a probable cause of the alignment-broken effect. Inspired by our
findings, we propose Vaccine, a perturbation-aware alignment technique to
mitigate the security risk of users finetuning. The core idea of Vaccine is to
produce invariant hidden embeddings by progressively adding crafted
perturbation to them in the alignment phase. This enables the embeddings to
withstand harmful perturbation from un-sanitized user data in the finetuning
phase. Our results on open source mainstream LLMs (e.g., Llama2, Opt, Vicuna)
demonstrate that Vaccine can boost the robustness of alignment against harmful
prompts induced embedding drift while reserving reasoning ability towards
benign prompts. Our code is available at
https://github.com/git-disl/Vaccine.

新型的定型服务范式引入了大型语言模型（LLMs）的新攻击面：用户上传的少量有害数据可以轻易地欺骗定型，从而产生对齐破坏的模型。我们进行了实证分析，揭示了一种可能引起对齐破坏效应的有害嵌入漂移现象。在我们的研究发现的启示下，我们提出了一种名为 Vaccine 的扰动感知对齐技术，以减轻用户定型的安全风险。疫苗的核心思想是通过在对齐阶段逐步添加精心设计的扰动，产生不变的隐藏嵌入。这使得嵌入能够抵御定型阶段不经过消毒处理的用户数据所带来的有害扰动。我们在开源主流 LLMs（例如 Llama2、Opt、Vicuna）上的实验结果表明，Vaccine 可以增强对抗由有害提示引起的嵌入漂移而保留对良性提示的推理能力。我们的代码可在 https://github.com/git-disl/Vaccine 获取。