Large Language Models (LLMs) are increasingly used in applications where the
model selects from competing third-party content, such as in LLM-powered search
engines or chatbot plugins. In this paper, we introduce Preference Manipulation
Attacks, a new class of attacks that manipulate an LLM's selections to favor
the attacker. We demonstrate that carefully crafted website content or plugin
documentations can trick an LLM to promote the attacker products and discredit
competitors, thereby increasing user traffic and monetization. We show this
leads to a prisoner's dilemma, where all parties are incentivized to launch
attacks, but the collective effect degrades the LLM's outputs for everyone. We
demonstrate our attacks on production LLM search engines (Bing and Perplexity)
and plugin APIs (for GPT-4 and Claude). As LLMs are increasingly used to rank
third-party content, we expect Preference Manipulation Attacks to emerge as a
significant threat.

大语言模型（LLM）被越来越多地应用于模型在选择竞争的第三方内容方面，比如 LLM 驱动的搜索引擎或聊天机器人插件。本文介绍了一种新的攻击类型 —— 偏好操纵攻击，可以通过精心编写的网站内容或插件文档来误导 LLM 以提升攻击者产品的地位并贬低竞争对手，从而增加用户流量和变现。我们展示了这种攻击导致囚徒困境，其中所有各方都有动机发动攻击，但是整体效果会降低 LLM 的输出质量。我们在生产环境的 LLM 搜索引擎（Bing 和 Perplexity）以及插件 API（用于 GPT-4 和 Claude）上演示了我们的攻击。随着 LLM 被越来越多地用于排列第三方内容，我们预计偏好操纵攻击将成为一个重大威胁。