Recent work has embodied LLMs as agents, allowing them to access tools,
perform actions, and interact with external content (e.g., emails or websites).
However, external content introduces the risk of indirect prompt injection
(IPI) attacks, where malicious instructions are embedded within the content
processed by LLMs, aiming to manipulate these agents into executing detrimental
actions against users. Given the potentially severe consequences of such
attacks, establishing benchmarks to assess and mitigate these risks is
imperative.
In this work, we introduce InjecAgent, a benchmark designed to assess the
vulnerability of tool-integrated LLM agents to IPI attacks. InjecAgent
comprises 1,054 test cases covering 17 different user tools and 62 attacker
tools. We categorize attack intentions into two primary types: direct harm to
users and exfiltration of private data. We evaluate 30 different LLM agents and
show that agents are vulnerable to IPI attacks, with ReAct-prompted GPT-4
vulnerable to attacks 24% of the time. Further investigation into an enhanced
setting, where the attacker instructions are reinforced with a hacking prompt,
shows additional increases in success rates, nearly doubling the attack success
rate on the ReAct-prompted GPT-4. Our findings raise questions about the
widespread deployment of LLM Agents. Our benchmark is available at
this https URL

通过引入 InjecAgent 基准测试，评估 LLM agents 对 IPI attacks 的脆弱性，结果显示 LLM agents 易受攻击，ReAct-prompted GPT-4 在 24% 的情况下易受攻击，并结合加强的黑客提示进一步提高攻击成功率，对 ReAct-prompted GPT-4 的攻击成功率几乎翻倍。