Large language models and AI chatbots have been at the forefront of
democratizing artificial intelligence. However, the releases of ChatGPT and
other similar tools have been followed by growing concerns regarding the
difficulty of controlling large language models and their outputs. Currently,
we are witnessing a cat-and-mouse game where users attempt to misuse the models
with a novel attack called prompt injections. In contrast, the developers
attempt to discover the vulnerabilities and block the attacks simultaneously.
In this paper, we provide an overview of these emergent threats and present a
categorization of prompt injections, which can guide future research on prompt
injections and act as a checklist of vulnerabilities in the development of LLM
interfaces. Moreover, based on previous literature and our own empirical
research, we discuss the implications of prompt injections to LLM end users,
developers, and researchers.

大语言模型和 AI 聊天机器人在使人工智能民主化方面处于前沿。然而，发布 ChatGPT 和其他类似工具后，人们越来越担心难以控制大语言模型及其输出的问题。目前，我们正目睹用户试图滥用这些模型而开展的一场猫鼠大战，新出现了一种名为提示注入的攻击方式。相反，开发人员试图同时发现这些漏洞并阻止攻击。在本文中，我们概述了这些新出现的威胁，并提供提示注入的分类，以指导未来有关提示注入的研究，并作为在 LLM 接口开发中漏洞检查清单。此外，基于先前的文献和我们自己的实证研究，我们还讨论了提示注入对 LLM 终端用户、开发人员和研究人员的影响。