Warning: This paper contains examples of harmful language and images, and
reader discretion is recommended. The security concerns surrounding Large
Language Models (LLMs) have been extensively explored, yet the safety of Large
Multi-Modal Models (LMMs) remains understudied. In our study, we present a
novel visual prompt attack that exploits query-relevant images to jailbreak the
open-source LMMs. Our method creates a composite image from one image generated
by diffusion models and another that displays the text as typography, based on
keywords extracted from a malicious query. We show LLMs can be easily attacked
by our approach, even if the employed Large Language Models are safely aligned.
To evaluate the extent of this vulnerability in open-source LMMs, we have
compiled a substantial dataset encompassing 13 scenarios with a total of 5,040
text-image pairs, using our presented attack technique. Our evaluation of 12
cutting-edge LMMs using this dataset shows the vulnerability of existing
multi-modal models on adversarial attacks. This finding underscores the need
for a concerted effort to strengthen and enhance the safety measures of
open-source LMMs against potential malicious exploits. The resource is
available at \href{this https URL}{this https URL}.

我们用一种新的视觉提示攻击方法，通过利用与查询相关的图像来越狱开源的大型多模型模型 (LMMs)。我们的研究表明，即使采用了安全对齐的大型语言模型，我们的方法也能轻松攻击 LLMs。通过使用我们提出的攻击技术，我们编制了一个大规模数据集，评估了 12 种先进的 LMMs 在 13 种情景中的弱点，这凸显了存在对抗性攻击的多模型模型的脆弱性。这一发现加强了我们对加强开源 LMMs 安全措施以防止潜在恶意利用的需求。