To ensure AI safety, instruction-tuned Large Language Models (LLMs) are
specifically trained to ensure alignment, which refers to making models behave
in accordance with human intentions. While these models have demonstrated
commendable results on various safety benchmarks, the vulnerability of their
safety alignment has not been extensively studied. This is particularly
troubling given the potential harm that LLMs can inflict. Existing attack
methods on LLMs often rely on poisoned training data or the injection of
malicious prompts. These approaches compromise the stealthiness and
generalizability of the attacks, making them susceptible to detection.
Additionally, these models often demand substantial computational resources for
implementation, making them less practical for real-world applications. In this
work, we introduce a novel attack framework, called Backdoor Activation Attack,
which injects trojan steering vectors into the activation layers of LLMs. These
malicious steering vectors can be triggered at inference time to steer the
models toward attacker-desired behaviors by manipulating their activations. In
particular, the steering vectors are generated by taking the difference between
benign and malicious activations. Then, the most effective steering vector is
selected and added to the forward passes of the LLMs. Our experiment results on
four primary alignment tasks show that our proposed method is highly effective
and adds little or no overhead to attack efficiency. Additionally, we discuss
potential countermeasures against such activation attacks. Our code and data
are available at this https URL Warning: this paper contains
content that can be offensive or upsetting.

通过向大型语言模型注入木马激活向量，我们提出了一种名为后门激活攻击的新型攻击框架，使得模型在推理时可以被激活并朝着攻击者所期望的行为方向进行操纵，该方法在主要的对齐任务上表现出高度的有效性，并且几乎不会给攻击效率增加任何开销，同时讨论了对抗此类激活攻击的潜在对策。