Today's LLMs are susceptible to prompt injections, jailbreaks, and other
attacks that allow adversaries to overwrite a model's original instructions
with their own malicious prompts. In this work, we argue that one of the
primary vulnerabilities underlying these attacks is that LLMs often consider
system prompts (e.g., text from an application developer) to be the same
priority as text from untrusted users and third parties. To address this, we
propose an instruction hierarchy that explicitly defines how models should
behave when instructions of different priorities conflict. We then propose a
data generation method to demonstrate this hierarchical instruction following
behavior, which teaches LLMs to selectively ignore lower-privileged
instructions. We apply this method to GPT-3.5, showing that it drastically
increases robustness -- even for attack types not seen during training -- while
imposing minimal degradations on standard capabilities.

今天的 LLMs 容易受到即时注入、越狱和其他攻击的影响，使得恶意提示可以覆盖模型的初始指令。本文提出一种指令层次结构，明确定义了在不同优先级指令冲突时模型应该如何行为，并提出了一种数据生成方法来展示这种层次指令遵循行为，教导 LLMs 有选择性地忽略低权限指令。我们将这种方法应用于 GPT-3.5 上，展示它显著增加了鲁棒性，甚至对训练期间未见的攻击类型，同时对标准能力的降低影响很小。