Visual State Space Model (VMamba) has recently emerged as a promising
architecture, exhibiting remarkable performance in various computer vision
tasks. However, its robustness has not yet been thoroughly studied. In this
paper, we delve into the robustness of this architecture through comprehensive
investigations from multiple perspectives. Firstly, we investigate its
robustness to adversarial attacks, employing both whole-image and
patch-specific adversarial attacks. Results demonstrate superior adversarial
robustness compared to Transformer architectures while revealing scalability
weaknesses. Secondly, the general robustness of VMamba is assessed against
diverse scenarios, including natural adversarial examples, out-of-distribution
data, and common corruptions. VMamba exhibits exceptional generalizability with
out-of-distribution data but shows scalability weaknesses against natural
adversarial examples and common corruptions. Additionally, we explore VMamba's
gradients and back-propagation during white-box attacks, uncovering unique
vulnerabilities and defensive capabilities of its novel components. Lastly, the
sensitivity of VMamba to image structure variations is examined, highlighting
vulnerabilities associated with the distribution of disturbance areas and
spatial information, with increased susceptibility closer to the image center.
Through these comprehensive studies, we contribute to a deeper understanding of
VMamba's robustness, providing valuable insights for refining and advancing the
capabilities of deep neural networks in computer vision applications.

通过多个角度对 Visual State Space Model (VMamba) 的鲁棒性进行全面研究，揭示了其在对抗性攻击、一般鲁棒性、梯度及反向传播过程、图像结构变化等方面的独特弱点和防御能力，为促进计算机视觉应用中深度神经网络的能力提供了宝贵的洞见。

理解用于图像分类的视觉状态空间模型的鲁棒性

Understanding Robustness of Visual State Space Models for Image  Classification

Convolutional Neural Networks (CNNs) and Vision Transformers (ViTs) stand as
the two most popular foundation models for visual representation learning.
While CNNs exhibit remarkable scalability with linear complexity w.r.t. image
resolution, ViTs surpass them in fitting capabilities despite contending with
quadratic complexity. A closer inspection reveals that ViTs achieve superior
visual modeling performance through the incorporation of global receptive
fields and dynamic weights. This observation motivates us to propose a novel
architecture that inherits these components while enhancing computational
efficiency. To this end, we draw inspiration from the recently introduced state
space model and propose the Visual State Space Model (VMamba), which achieves
linear complexity without sacrificing global receptive fields. To address the
encountered direction-sensitive issue, we introduce the Cross-Scan Module (CSM)
to traverse the spatial domain and convert any non-causal visual image into
order patch sequences. Extensive experimental results substantiate that VMamba
not only demonstrates promising capabilities across various visual perception
tasks, but also exhibits more pronounced advantages over established benchmarks
as the image resolution increases. Source code has been available at
this https URL

基于卷积神经网络和视觉变换器，提出了一种名为 Visual State Space Model (VMamba) 的新型体系结构，具有线性复杂度和全局感受野，并在图像分辨率增加时展现出更明显的优势。