With the fast development of large language models (LLMs), LLM-driven Web Agents (Web Agents for short) have obtained tons of attention due to their superior capability where LLMs serve as the core part of making decisions like the human brain equipped with multiple web tools to actively interact with external deployed websites. As uncountable Web Agents have been released and such LLM systems are experiencing rapid development and drawing closer to widespread deployment in our daily lives, an essential and pressing question arises: "Are these Web Agents secure?". In this paper, we introduce a novel threat, WIPI, that indirectly controls Web Agent to execute malicious instructions embedded in publicly accessible webpages. To launch a successful WIPI works in a black-box environment. This methodology focuses on the form and content of indirect instructions within external webpages, enhancing the efficiency and stealthiness of the attack. To evaluate the effectiveness of the proposed methodology, we conducted extensive experiments using 7 plugin-based ChatGPT Web Agents, 8 Web GPTs, and 3 different open-source Web Agents. The results reveal that our methodology achieves an average attack success rate (ASR) exceeding 90% even in pure black-box scenarios. Moreover, through an ablation study examining various user prefix instructions, we demonstrated that the WIPI exhibits strong robustness, maintaining high performance across diverse prefix instructions.

LLM驱动的Web Agents（英语简写为Web Agents）因其在像人脑一样通过多种网络工具与外部部署的网站进行积极互动时，具备强大的能力而备受关注。本文介绍了一种名为WIPI的新型威胁，其通过间接控制Web Agents在公开的网页中执行恶意指令。实验证明，即便在纯黑盒场景中，该方法达到了超过90%的平均攻击成功率（ASR），并且在多样化的前缀指令情况下仍具有高性能和强大的鲁棒性。

WIPI：基于LLM的网络代理新威胁