Modern spacecraft are increasingly relying on machine learning (ML). However,
physical equipment in space is subject to various natural hazards, such as
radiation, which may inhibit the correct operation of computing devices.
Despite plenty of evidence showing the damage that naturally-induced faults can
cause to ML-related hardware, we observe that the effects of radiation on ML
models for space applications are not well-studied. This is a problem: without
understanding how ML models are affected by these natural phenomena, it is
uncertain "where to start from" to develop radiation-tolerant ML software. As
ML researchers, we attempt to tackle this dilemma. By partnering up with
space-industry practitioners specialized in ML, we perform a reflective
analysis of the state of the art. We provide factual evidence that prior work
did not thoroughly examine the impact of natural hazards on ML models meant for
spacecraft. Then, through a "negative result", we show that some existing
open-source technologies can hardly be used by researchers to study the effects
of radiation for some applications of ML in satellites. As a constructive step
forward, we perform simple experiments showcasing how to leverage current
frameworks to assess the robustness of practical ML models for cloud detection
against radiation-induced faults. Our evaluation reveals that not all faults
are as devastating as claimed by some prior work. By publicly releasing our
resources, we provide a foothold -- usable by researchers without access to
spacecraft -- for spearheading development of space-tolerant ML models.

现代航天器越来越依赖于机器学习，然而，在太空中的物理设备会受到不同的自然灾害的影响，例如辐射，这可能会阻碍计算设备的正确运行。本文呈现了机器学习模型在太空应用中受到自然现象影响的研究进展，以及开放资源的有效性，从而推动了航天容忍机器学习模型的发展。

空间机器学习：评估机载机器学习模型对辐射的稳健性

Machine Learning in Space: Surveying the Robustness of on-board ML  models to Radiation

We explore the implementation of deep learning techniques for precise
building damage assessment in the context of natural hazards, utilizing remote
sensing data. The xBD dataset, comprising diverse disaster events from across
the globe, serves as the primary focus, facilitating the evaluation of deep
learning models. We tackle the challenges of generalization to novel disasters
and regions while accounting for the influence of low-quality and noisy labels
inherent in natural hazard data. Furthermore, our investigation quantitatively
establishes that the minimum satellite imagery resolution essential for
effective building damage detection is 3 meters and below 1 meter for
classification using symmetric and asymmetric resolution perturbation analyses.
To achieve robust and accurate evaluations of building damage detection and
classification, we evaluated different deep learning models with residual,
squeeze and excitation, and dual path network backbones, as well as ensemble
techniques. Overall, the U-Net Siamese network ensemble with F-1 score of 0.812
performed the best against the xView2 challenge benchmark. Additionally, we
evaluate a Universal model trained on all hazards against a flood expert model
and investigate generalization gaps across events, and out of distribution from
field data in the Ahr Valley. Our research findings showcase the potential and
limitations of advanced AI solutions in enhancing the impact assessment of
climate change-induced extreme weather events, such as floods and hurricanes.
These insights have implications for disaster impact assessment in the face of
escalating climate challenges.

通过使用深度学习技术来精确评估自然灾害中建筑损害，在利用遥感数据的背景下进行实现。我们探索使用来自全球各地的多样化灾难事件的 xBD 数据集作为主要焦点，以评估深度学习模型。我们解决了推广到新灾难和地区的挑战，同时考虑了自然灾害数据中固有的低质量和噪声标签的影响。此外，我们的研究定量地证明了对于有效的建筑损害检测，最低的卫星图像分辨率为 3 米，对于使用对称和非对称分辨率摄动分析的分类低于 1 米。为了实现对建筑损害检测和分类的稳健准确评估，我们评估了具有残差、压缩和激活以及双路径网络主干的不同深度学习模型，以及集成技术。总体而言，F-1 得分为 0.812 的 U-Net Siamese 网络集成在 xView2 挑战基准测试中表现最好。此外，我们还将对所有灾害进行训练的通用模型与洪水专家模型进行评估，并调查了在 Ahr Valley 的领域数据中的事件间通用性差距和分布。我们的研究发现展示了先进人工智能解决方案在增强气候变化引发的极端天气事件（如洪水和飓风）影响评估方面的潜力和局限性。这些见解对于在不断升级的气候挑战面前的灾害影响评估具有重要意义。