Recent developments in Large Language Models (LLMs) have manifested
significant advancements. To facilitate safeguards against malicious
exploitation, a body of research has concentrated on aligning LLMs with human
preferences and inhibiting their generation of inappropriate content.
Unfortunately, such alignments are often vulnerable: fine-tuning with a minimal
amount of harmful data can easily unalign the target LLM. While being
effective, such fine-tuning-based unalignment approaches also have their own
limitations: (1) non-stealthiness, after fine-tuning, safety audits or
red-teaming can easily expose the potential weaknesses of the unaligned models,
thereby precluding their release/use. (2) non-persistence, the unaligned LLMs
can be easily repaired through re-alignment, i.e., fine-tuning again with
aligned data points. In this work, we show that it is possible to conduct
stealthy and persistent unalignment on large language models via backdoor
injections. We also provide a novel understanding on the relationship between
the backdoor persistence and the activation pattern and further provide
guidelines for potential trigger design. Through extensive experiments, we
demonstrate that our proposed stealthy and persistent unalignment can
successfully pass the safety evaluation while maintaining strong persistence
against re-alignment defense.

通过背门注入，在大型语言模型上进行隐秘而持续的非对准操作，可以成功通过安全评估，同时对抗重新对齐防御。