While vision-language pre-training model (VLP) has shown revolutionary
improvements on various vision-language (V+L) tasks, the studies regarding its
adversarial robustness remain largely unexplored. This paper studied the
adversarial attack on popular VLP models and V+L tasks. First, we analyzed the
performance of adversarial attacks under different settings. By examining the
influence of different perturbed objects and attack targets, we concluded some
key observations as guidance on both designing strong multimodal adversarial
attack and constructing robust VLP models. Second, we proposed a novel
multimodal attack method on the VLP models called Collaborative Multimodal
Adversarial Attack (Co-Attack), which collectively carries out the attacks on
the image modality and the text modality. Experimental results demonstrated
that the proposed method achieves improved attack performances on different V+L
downstream tasks and VLP models. The analysis observations and novel attack
method hopefully provide new understanding into the adversarial robustness of
VLP models, so as to contribute their safe and reliable deployment in more
real-world scenarios. Code is available at
this https URL.

本研究针对视觉 - 语言预训练模型（VLP）的鲁棒性问题进行研究，提出了一种名为协作多模态对抗攻击（Co-Attack）的新型攻击方法，以获得对不同视觉 - 语言下游任务和 VLP 模型的更好攻击性能。该研究结果提供了关于 VLP 模型的对抗鲁棒性的新认识，为其在更多实际场景中的安全可靠部署做出了贡献。