This paper focuses on a Vision-based Landing task and presents the design and
the validation of a dataset that would comply with the Operational Design
Domain (ODD) of a Machine-Learning (ML) system. Relying on emerging
certification standards, we describe the process for establishing ODDs at both
the system and image levels. In the process, we present the translation of
high-level system constraints into actionable image-level properties, allowing
for the definition of verifiable Data Quality Requirements (DQRs). To
illustrate this approach, we use the Landing Approach Runway Detection (LARD)
dataset which combines synthetic imagery and real footage, and we focus on the
steps required to verify the DQRs. The replicable framework presented in this
paper addresses the challenges of designing a dataset compliant with the
stringent needs of ML-based systems certification in safety-critical
applications.

本研究聚焦于基于视觉的着陆任务，展示了符合机器学习系统操作设计领域（ODD）的数据集的设计和验证。通过借鉴新兴的认证标准，我们描述了在系统和图像级别建立 ODDs 的流程。在该过程中，我们将高级系统约束转化为可操作的图像级属性，从而定义了可验证的数据质量要求 (DQRs)。为了阐述这种方法，我们使用了 Landing Approach Runway Detection (LARD) 数据集，结合了合成图像和真实画面，并侧重于验证 DQRs 所需的步骤。本论文中提出的可复制框架解决了为安全关键应用中基于 ML 的系统认证设计符合严格需求的挑战。

如何设计符合基于机器学习系统的 ODD 的数据集

How to design a dataset compliant with an ML-based system ODD?

Randomized smoothing (RS) has successfully been used to improve the
robustness of predictions for deep neural networks (DNNs) by adding random
noise to create multiple variations of an input, followed by deciding the
consensus. To understand if an RS-enabled DNN is effective in the sampled input
domains, it is mandatory to sample data points within the operational design
domain, acquire the point-wise certificate regarding robustness radius, and
compare it with pre-defined acceptance criteria. Consequently, ensuring that a
point-wise robustness certificate for any given data point is obtained
relatively cost-effectively is crucial. This work demonstrates that reducing
the number of samples by one or two orders of magnitude can still enable the
computation of a slightly smaller robustness radius (commonly ~20% radius
reduction) with the same confidence. We provide the mathematical foundation for
explaining the phenomenon while experimentally showing promising results on the
standard CIFAR-10 and ImageNet datasets.

减少样本数量可以降低鲁棒性半径，但保持相同的置信度；本文提供了数学基础并在 CIFAR-10 和 ImageNet 数据集上进行了实验证明。

使用 100 倍的样本效率估计随机平滑的稳健半径

Estimating the Robustness Radius for Randomized Smoothing with  100$\times$ Sample Efficiency

Modern AI techniques open up ever-increasing possibilities for autonomous
vehicles, but how to appropriately verify the reliability of such systems
remains unclear. A common approach is to conduct safety validation based on a
predefined Operational Design Domain (ODD) describing specific conditions under
which a system under test is required to operate properly. However, collecting
sufficient realistic test cases to ensure comprehensive ODD coverage is
challenging. In this paper, we report our practical experiences regarding the
utility of data simulation with deep generative models for scenario-based ODD
validation. We consider the specific use case of a camera-based rail-scene
segmentation system designed to support autonomous train operation. We
demonstrate the capabilities of semantically editing railway scenes with deep
generative models to make a limited amount of test data more representative. We
also show how our approach helps to analyze the degree to which a system
complies with typical ODD requirements. Specifically, we focus on evaluating
proper operation under different lighting and weather conditions as well as
while transitioning between them.

利用深度生成模型的数据模拟实用性来对场景 - based ODD 进行验证，并分析系统在不同光照和天气条件下以及在其之间过渡时的运作程度。