Large Language Model (LLM) systems are inherently compositional, with
individual LLM serving as the core foundation with additional layers of objects
such as plugins, sandbox, and so on. Along with the great potential, there are
also increasing concerns over the security of such probabilistic intelligent
systems. However, existing studies on LLM security often focus on individual
LLM, but without examining the ecosystem through the lens of LLM systems with
other objects (e.g., Frontend, Webtool, Sandbox, and so on). In this paper, we
systematically analyze the security of LLM systems, instead of focusing on the
individual LLMs. To do so, we build on top of the information flow and
formulate the security of LLM systems as constraints on the alignment of the
information flow within LLM and between LLM and other objects. Based on this
construction and the unique probabilistic nature of LLM, the attack surface of
the LLM system can be decomposed into three key components: (1) multi-layer
security analysis, (2) analysis of the existence of constraints, and (3)
analysis of the robustness of these constraints. To ground this new attack
surface, we propose a multi-layer and multi-step approach and apply it to the
state-of-art LLM system, OpenAI GPT4. Our investigation exposes several
security issues, not just within the LLM model itself but also in its
integration with other components. We found that although the OpenAI GPT4 has
designed numerous safety constraints to improve its safety features, these
safety constraints are still vulnerable to attackers. To further demonstrate
the real-world threats of our discovered vulnerabilities, we construct an
end-to-end attack where an adversary can illicitly acquire the user's chat
history, all without the need to manipulate the user's input or gain direct
access to OpenAI GPT4. Our demo is in the link:
this https URL

在本文中，我们系统地分析了 LLM 系统的安全性，通过建立信息流及其在 LLM 内部和 LLM 与其他对象之间的一致性约束，将 LLM 系统的攻击面分解为三个关键部分：多层安全分析，约束存在性分析和这些约束的鲁棒性分析。通过对最新的 LLM 系统 OpenAI GPT4 的多层和多步骤分析，我们发现了不仅仅存在于 LLM 模型本身，还包括与其他组件整合的安全问题。我们发现尽管 OpenAI GPT4 已经设计了多重安全约束来提高其安全性，但这些安全约束仍然容易受到攻击。为了进一步展示我们发现的漏洞的现实威胁，我们构建了一个端到端攻击，黑客可以非法获取用户的聊天历史，而无需操纵用户的输入或直接访问 OpenAI GPT4。