Large Language Models (LLMs) have become widely adopted recently. Research
explores their use both as autonomous agents and as tools for software
engineering. LLM-integrated applications, on the other hand, are software
systems that leverage an LLM to perform tasks that would otherwise be
impossible or require significant coding effort. While LLM-integrated
application engineering is emerging as new discipline, its terminology,
concepts and methods need to be established. This study provides a taxonomy for
LLM-integrated applications, offering a framework for analyzing and describing
these systems. It also demonstrates various ways to utilize LLMs in
applications, as well as options for implementing such integrations.
Following established methods, we analyze a sample of recent LLM-integrated
applications to identify relevant dimensions. We evaluate the taxonomy by
applying it to additional cases. This review shows that applications integrate
LLMs in numerous ways for various purposes. Frequently, they comprise multiple
LLM integrations, which we term ``LLM components''. To gain a clear
understanding of an application's architecture, we examine each LLM component
separately. We identify thirteen dimensions along which to characterize an LLM
component, including the LLM skills leveraged, the format of the output, and
more. LLM-integrated applications are described as combinations of their LLM
components. We suggest a concise representation using feature vectors for
visualization.
The taxonomy is effective for describing LLM-integrated applications. It can
contribute to theory building in the nascent field of LLM-integrated
application engineering and aid in developing such systems. Researchers and
practitioners explore numerous creative ways to leverage LLMs in applications.
Though challenges persist, integrating LLMs may revolutionize the way software
systems are built.

大型语言模型与应用集成研究的分类体系和 LLM 组件的描述，以及它们在软件系统建设方面的潜力。

大型语言模型作为软件组件：LLM 集成应用的分类方法

Large Language Models as Software Components: A Taxonomy for  LLM-Integrated Applications

With the proliferation of LLM-integrated applications such as GPT-s, millions
are deployed, offering valuable services through proprietary instruction
prompts. These systems, however, are prone to prompt extraction attacks through
meticulously designed queries. To help mitigate this problem, we introduce the
Raccoon benchmark which comprehensively evaluates a model's susceptibility to
prompt extraction attacks. Our novel evaluation method assesses models under
both defenseless and defended scenarios, employing a dual approach to evaluate
the effectiveness of existing defenses and the resilience of the models. The
benchmark encompasses 14 categories of prompt extraction attacks, with
additional compounded attacks that closely mimic the strategies of potential
attackers, alongside a diverse collection of defense templates. This array is,
to our knowledge, the most extensive compilation of prompt theft attacks and
defense mechanisms to date. Our findings highlight universal susceptibility to
prompt theft in the absence of defenses, with OpenAI models demonstrating
notable resilience when protected. This paper aims to establish a more
systematic benchmark for assessing LLM robustness against prompt extraction
attacks, offering insights into their causes and potential countermeasures.
Resources of Raccoon are publicly available at
this https URL

介绍了一种全面评估模型对提示提取攻击易感性的新方法和基准测试，以及对现有防御和模型韧性的有效性进行评估，为评估 LLM 对提示提取攻击的鲁棒性建立了更系统的基准测试，并提出了攻击原因和潜在对策。

Raccoon: LLM 集成应用的提示提取基准

Raccoon: Prompt Extraction Benchmark of LLM-Integrated Applications

Large language models (LLMs) are increasingly deployed as the service backend
for LLM-integrated applications such as code completion and AI-powered search.
LLM-integrated applications serve as middleware to refine users' queries with
domain-specific knowledge to better inform LLMs and enhance the responses.
Despite numerous opportunities and benefits, LLM-integrated applications also
introduce new attack surfaces. Understanding, minimizing, and eliminating these
emerging attack surfaces is a new area of research. In this work, we consider a
setup where the user and LLM interact via an LLM-integrated application in the
middle. We focus on the communication rounds that begin with user's queries and
end with LLM-integrated application returning responses to the queries, powered
by LLMs at the service backend. For this query-response protocol, we identify
potential vulnerabilities that can originate from the malicious application
developer or from an outsider threat initiator that is able to control the
database access, manipulate and poison data that are high-risk for the user.
Successful exploits of the identified vulnerabilities result in the users
receiving responses tailored to the intent of a threat initiator. We assess
such threats against LLM-integrated applications empowered by OpenAI GPT-3.5
and GPT-4. Our empirical results show that the threats can effectively bypass
the restrictions and moderation policies of OpenAI, resulting in users
receiving responses that contain bias, toxic content, privacy risk, and
disinformation. To mitigate those threats, we identify and define four key
properties, namely integrity, source identification, attack detectability, and
utility preservation, that need to be satisfied by a safe LLM-integrated
application. Based on these properties, we develop a lightweight,
threat-agnostic defense that mitigates both insider and outsider threats.

使用大型语言模型（LLMs）作为 LLM 集成应用程序的服务后端的情况下，我们发现了潜在的攻击面和漏洞，并开发了一种轻量级、抗威胁的防御方法，以最大程度减少威胁并保证应用程序的完整性、来源识别、攻击可检测性和实用性保留。