Vision-Large-Language-Models (Vision-LLMs) are increasingly being integrated
into autonomous driving (AD) systems due to their advanced visual-language
reasoning capabilities, targeting the perception, prediction, planning, and
control mechanisms. However, Vision-LLMs have demonstrated susceptibilities
against various types of adversarial attacks, which would compromise their
reliability and safety. To further explore the risk in AD systems and the
transferability of practical threats, we propose to leverage typographic
attacks against AD systems relying on the decision-making capabilities of
Vision-LLMs. Different from the few existing works developing general datasets
of typographic attacks, this paper focuses on realistic traffic scenarios where
these attacks can be deployed, on their potential effects on the
decision-making autonomy, and on the practical ways in which these attacks can
be physically presented. To achieve the above goals, we first propose a
dataset-agnostic framework for automatically generating false answers that can
mislead Vision-LLMs' reasoning. Then, we present a linguistic augmentation
scheme that facilitates attacks at image-level and region-level reasoning, and
we extend it with attack patterns against multiple reasoning tasks
simultaneously. Based on these, we conduct a study on how these attacks can be
realized in physical traffic scenarios. Through our empirical study, we
evaluate the effectiveness, transferability, and realizability of typographic
attacks in traffic scenes. Our findings demonstrate particular harmfulness of
the typographic attacks against existing Vision-LLMs (e.g., LLaVA, Qwen-VL,
VILA, and Imp), thereby raising community awareness of vulnerabilities when
incorporating such models into AD systems. We will release our source code upon
acceptance.

借助于大规模视觉 - 语言模型，本研究探索了针对自动驾驶系统的印刷攻击，重点关注真实交通场景中这些攻击的部署方式及其对决策自主性的潜在影响，对于已有的大规模视觉 - 语言模型表示特别有害的印刷攻击引起了社区的关注。

面向自动驾驶中基于视觉和语言模型的可转移攻击的研究

Towards Transferable Attacks Against Vision-LLMs in Autonomous Driving  with Typography

Recently, significant progress has been made on Large Vision-Language Models
(LVLMs); a new class of VL models that make use of large pre-trained language
models. Yet, their vulnerability to Typographic attacks, which involve
superimposing misleading text onto an image remain unstudied. Furthermore,
prior work typographic attacks rely on sampling a random misleading class from
a predefined set of classes. However, the random chosen class might not be the
most effective attack. To address these issues, we first introduce a novel
benchmark uniquely designed to test LVLMs vulnerability to typographic attacks.
Furthermore, we introduce a new and more effective typographic attack:
Self-Generated typographic attacks. Indeed, our method, given an image, make
use of the strong language capabilities of models like GPT-4V by simply
prompting them to recommend a typographic attack. Using our novel benchmark, we
uncover that typographic attacks represent a significant threat against
LVLM(s). Furthermore, we uncover that typographic attacks recommended by GPT-4V
using our new method are not only more effective against GPT-4V itself compared
to prior work attacks, but also against a host of less capable yet popular open
source models like LLaVA, InstructBLIP, and MiniGPT4.

大视觉 - 语言模型（LVLMs）对印刷攻击的脆弱性进行了研究，并引入了一种新的基准测试和一种更有效的自动生成印刷攻击方法。

视觉伪造技术 LLMs 自我生成的字体攻击

Vision-LLMs Can Fool Themselves with Self-Generated Typographic Attacks

The CLIP (Contrastive Language-Image Pre-training) model and its variants are
becoming the de facto backbone in many applications. However, training a CLIP
model from hundreds of millions of image-text pairs can be prohibitively
expensive. Furthermore, the conventional CLIP model doesn't differentiate
between the visual semantics and meaning of text regions embedded in images.
This can lead to non-robustness when the text in the embedded region doesn't
match the image's visual appearance. In this paper, we discuss two effective
approaches to improve the efficiency and robustness of CLIP training: (1)
augmenting the training dataset while maintaining the same number of
optimization steps, and (2) filtering out samples that contain text regions in
the image. By doing so, we significantly improve the classification and
retrieval accuracy on public benchmarks like ImageNet and CoCo. Filtering out
images with text regions also protects the model from typographic attacks. To
verify this, we build a new dataset named ImageNet with Adversarial Text
Regions (ImageNet-Attr). Our filter-based CLIP model demonstrates a top-1
accuracy of 68.78\%, outperforming previous models whose accuracy was all below
50\%.

本文介绍两种有效方法来提高 CLIP 模型的效率和鲁棒性：（1）在维持相同优化步数的情况下增加训练数据集，（2）过滤包含图像中文本区域的样本，从而在像 ImageNet 和 CoCo 公共基准测试中显著提高分类和检索精度。过滤掉包含文本区域的图像还可保护模型免受排版攻击。通过构建名为 ImageNet with Adversarial Text Regions（ImageNet-Attr）的新数据集，验证了这一点。我们的基于过滤的 CLIP 模型在 top-1 精度方面表现出 68.78％的性能，胜过以前的所有模型，其精度均低于 50％。

少即是多：去除文本区域提高 CLIP 训练效率和鲁棒性

Less is More: Removing Text-regions Improves CLIP Training Efficiency  and Robustness

Vision-language pre-training models (VLPs) have exhibited revolutionary
improvements in various vision-language tasks. In VLP, some adversarial attacks
fool a model into false or absurd classifications. Previous studies addressed
these attacks by fine-tuning the model or changing its architecture. However,
these methods risk losing the original model's performance and are difficult to
apply to downstream tasks. In particular, their applicability to other tasks
has not been considered. In this study, we addressed the reduction of the
impact of typographic attacks on CLIP without changing the model parameters. To
achieve this, we expand the idea of ``prefix learning'' and introduce our
simple yet effective method: Defense-Prefix (DP), which inserts the DP token
before a class name to make words ``robust'' against typographic attacks. Our
method can be easily applied to downstream tasks, such as object detection,
because the proposed method is independent of the model parameters. Our method
significantly improves the accuracy of classification tasks for typographic
attack datasets, while maintaining the zero-shot capabilities of the model. In
addition, we leverage our proposed method for object detection, demonstrating
its high applicability and effectiveness. The codes and datasets will be
publicly available.

本研究提出了一种名为 Defense-Prefix（DP）的方法来防范视觉语言预训练模型中的排版攻击，该方法可以在不改变模型参数的情况下应用于目标检测等下游任务，并显著提高了分类任务的准确性。