Presently, with the assistance of advanced LLM application development
frameworks, more and more LLM-powered applications can effortlessly augment the
LLMs' knowledge with external content using the retrieval augmented generation
(RAG) technique. However, these frameworks' designs do not have sufficient
consideration of the risk of external content, thereby allowing attackers to
undermine the applications developed with these frameworks. In this paper, we
reveal a new threat to LLM-powered applications, termed retrieval poisoning,
where attackers can guide the application to yield malicious responses during
the RAG process. Specifically, through the analysis of LLM application
frameworks, attackers can craft documents visually indistinguishable from
benign ones. Despite the documents providing correct information, once they are
used as reference sources for RAG, the application is misled into generating
incorrect responses. Our preliminary experiments indicate that attackers can
mislead LLMs with an 88.33\% success rate, and achieve a 66.67\% success rate
in the real-world application, demonstrating the potential impact of retrieval
poisoning.

LLM 应用开发、检索增强生成、LLM 应用、检索污染以及风险评估是本文的关键词。作者揭示了一种称为检索污染的新威胁，攻击者可以通过欺骗 LLM 应用程序在检索生成过程中生成恶意回应，对应用程序进行破坏。通过分析 LLM 应用程序框架，攻击者可以制作与正常文档在视觉上几乎无法区分的文档，一旦这些文档被用作检索增强生成的参考来源，应用程序就会产生错误的响应。初步实验表明攻击者可以以 88.33% 的成功率误导 LLM，并在现实世界的应用中达到 66.67% 的成功率，展示了检索污染的潜在影响。