Large Multimodal Models (LMMs) have demonstrated the ability to interact with humans under real-world conditions by combining Large Language Models (LLMs) and modality encoders to align multimodal information (visual and auditory) with text. However, such models raise new safety challenges of whether models that are safety-aligned on text also exhibit consistent safeguards for multimodal inputs. Despite recent safety-alignment research on vision LMMs, the safety of audio LMMs remains under-explored. In this work, we comprehensively red team the safety of five advanced audio LMMs under three settings: (i) harmful questions in both audio and text formats, (ii) harmful questions in text format accompanied by distracting non-speech audio, and (iii) speech-specific jailbreaks. Our results under these settings demonstrate that open-source audio LMMs suffer an average attack success rate of 69.14% on harmful audio questions, and exhibit safety vulnerabilities when distracted with non-speech audio noise. Our speech-specific jailbreaks on Gemini-1.5-Pro achieve an attack success rate of 70.67% on the harmful query benchmark. We provide insights on what could cause these reported safety-misalignments. Warning: this paper contains offensive examples.

本研究针对大型多模态模型（LMMs）在音频方面的安全性问题进行了深入探讨，填补了目前在音频 LMMs 安全性研究中的空白。通过对五种先进音频 LMMs 进行全面的红队测试，结果显示它们在处理有害音频问题时攻击成功率高达 69.14%。该研究揭示了音频 LMMs 在安全对齐方面的脆弱性及其潜在原因，具有重要的安全性研究价值。 

音频是致命弱点：对音频大型多模态模型的红队测试