Despite the remarkable performance of video-based large language models
(LLMs), their adversarial threat remains unexplored. To fill this gap, we
propose the first adversarial attack tailored for video-based LLMs by crafting
flow-based multi-modal adversarial perturbations on a small fraction of frames
within a video, dubbed FMM-Attack. Extensive experiments show that our attack
can effectively induce video-based LLMs to generate incorrect answers when
videos are added with imperceptible adversarial perturbations. Intriguingly,
our FMM-Attack can also induce garbling in the model output, prompting
video-based LLMs to hallucinate. Overall, our observations inspire a further
understanding of multi-modal robustness and safety-related feature alignment
across different modalities, which is of great importance for various large
multi-modal models. Our code is available at
this https URL

使用流式多模态对抗性扰动，我们提出了首个专为基于视频的大型语言模型设计的对抗性攻击，有效地诱使此类模型生成错误答案，并引发幻觉。这一研究对多模态稳健性和安全相关特性对齐方面的进一步理解具有重要意义。