In overhead image segmentation tasks, including additional spectral bands
beyond the traditional RGB channels can improve model performance. However, it
is still unclear how incorporating this additional data impacts model
robustness to adversarial attacks and natural perturbations. For adversarial
robustness, the additional information could improve the model's ability to
distinguish malicious inputs, or simply provide new attack avenues and
vulnerabilities. For natural perturbations, the additional information could
better inform model decisions and weaken perturbation effects or have no
significant influence at all. In this work, we seek to characterize the
performance and robustness of a multispectral (RGB and near infrared) image
segmentation model subjected to adversarial attacks and natural perturbations.
While existing adversarial and natural robustness research has focused
primarily on digital perturbations, we prioritize on creating realistic
perturbations designed with physical world conditions in mind. For adversarial
robustness, we focus on data poisoning attacks whereas for natural robustness,
we focus on extending ImageNet-C common corruptions for fog and snow that
coherently and self-consistently perturbs the input data. Overall, we find both
RGB and multispectral models are vulnerable to data poisoning attacks
regardless of input or fusion architectures and that while physically
realizable natural perturbations still degrade model performance, the impact
differs based on fusion architecture and input data.

本文旨在研究一种多光谱（RGB 和近红外）图像分割模型在遭受对抗攻击和自然扰动时的性能和鲁棒性，并发现，无论是 RGB 模型还是多光谱模型都容易受到数据污染攻击的影响。然而，自然扰动的影响取决于融合架构和输入数据的不同。

深度多光谱分割模型对自然扰动和数据污染的鲁棒性量化

Quantifying the robustness of deep multispectral segmentation models  against natural perturbations and data poisoning

This work addresses the task of overhead image segmentation when auxiliary
ground-level images are available. Recent work has shown that performing joint
inference over these two modalities, often called near/remote sensing, can
yield significant accuracy improvements. Extending this line of work, we
introduce the concept of geospatial attention, a geometry-aware attention
mechanism that explicitly considers the geospatial relationship between the
pixels in a ground-level image and a geographic location. We propose an
approach for computing geospatial attention that incorporates geometric
features and the appearance of the overhead and ground-level imagery. We
introduce a novel architecture for near/remote sensing that is based on
geospatial attention and demonstrate its use for five segmentation tasks. The
results demonstrate that our method significantly outperforms the previous
state-of-the-art methods.

介绍了一种基于大地空间注意力机制的近 / 远距离感知方法，该方法结合了空中和地面图像的几何特征和外观，提高了分割任务的准确性。实验结果显示，该方法显著优于之前的最先进方法。