Vision State Space Models (VSSMs), a novel architecture that combines the
strengths of recurrent neural networks and latent variable models, have
demonstrated remarkable performance in visual perception tasks by efficiently
capturing long-range dependencies and modeling complex visual dynamics.
However, their robustness under natural and adversarial perturbations remains a
critical concern. In this work, we present a comprehensive evaluation of VSSMs'
robustness under various perturbation scenarios, including occlusions, image
structure, common corruptions, and adversarial attacks, and compare their
performance to well-established architectures such as transformers and
Convolutional Neural Networks. Furthermore, we investigate the resilience of
VSSMs to object-background compositional changes on sophisticated benchmarks
designed to test model performance in complex visual scenes. We also assess
their robustness on object detection and segmentation tasks using corrupted
datasets that mimic real-world scenarios. To gain a deeper understanding of
VSSMs' adversarial robustness, we conduct a frequency analysis of adversarial
attacks, evaluating their performance against low-frequency and high-frequency
perturbations. Our findings highlight the strengths and limitations of VSSMs in
handling complex visual corruptions, offering valuable insights for future
research and improvements in this promising field. Our code and models will be
available at this https URL

综合评估了视觉状态空间模型（VSSMs）在各种扰动场景下的稳健性，并与传统的架构如转换器和卷积神经网络进行了比较，发现其在处理复杂视觉破坏方面的优势和局限性，为该领域的未来研究和改进提供了宝贵的见解。