Large Visual Language Models (VLMs) such as GPT-4 have achieved remarkable success in generating comprehensive and nuanced responses, surpassing the capabilities of large language models. However, with the integration of visual inputs, new security concerns emerge, as malicious attackers can exploit multiple modalities to achieve their objectives. This has led to increasing attention on the vulnerabilities of VLMs to jailbreak. Most existing research focuses on generating adversarial images or nonsensical image collections to compromise these models. However, the challenge of leveraging meaningful images to produce targeted textual content using the VLMs' logical comprehension of images remains unexplored. In this paper, we explore the problem of logical jailbreak from meaningful images to text. To investigate this issue, we introduce a novel dataset designed to evaluate flowchart image jailbreak. Furthermore, we develop a framework for text-to-text jailbreak using VLMs. Finally, we conduct an extensive evaluation of the framework on GPT-4o and GPT-4-vision-preview, with jailbreak rates of 92.8% and 70.0%, respectively. Our research reveals significant vulnerabilities in current VLMs concerning image-to-text jailbreak. These findings underscore the need for a deeper examination of the security flaws in VLMs before their practical deployment.

使用大型视觉语言模型（VLMs）生成有深度和细致度的回复是非常成功的，然而，结合视觉输入后，新的安全隐患出现了，恶意攻击者可以利用多种模态来达到他们的目的。本文聚焦于通过有意义的图像来产生针对性文本，揭示了当前VLMs在图像转文本方面存在的严重漏洞，强调在实际部署之前需要对VLMs的安全漏洞进行深入研究。

图像到文本的逻辑越狱：你的想象力可以帮助你任何事情