With increasing complexity of Automated Driving Systems (ADS), ensuring their
safety and reliability has become a critical challenge. The Verification and
Validation (V&V) of these systems are particularly demanding when AI components
are employed to implement perception and/or control functions. In ESA-funded
project VIVAS, we developed a generic framework for system-level
simulation-based V&V of autonomous systems. The approach is based on a
simulation model of the system, an abstract model that describes symbolically
the system behavior, and formal methods to generate scenarios and verify the
simulation executions. Various coverage criteria can be defined to guide the
automated generation of the scenarios.
In this paper, we describe the instantiation of the VIVAS framework for an
ADS case study. This is based on the integration of CARLA, a widely-used
driving simulator, and its ScenarioRunner tool, which enables the creation of
diverse and complex driving scenarios. This is also used in the CARLA
Autonomous Driving Challenge to validate different ADS agents for perception
and control based on AI, shared by the CARLA community. We describe the
development of an abstract ADS model and the formulation of a coverage
criterion that focuses on the behaviors of vehicles relative to the vehicle
with ADS under verification. Leveraging the VIVAS framework, we generate and
execute various driving scenarios, thus testing the capabilities of the AI
components. The results show the effectiveness of VIVAS in automatically
generating scenarios for system-level simulation-based V&V of an automated
driving system using CARLA and ScenarioRunner. Therefore, they highlight the
potential of the approach as a powerful tool in the future of ADS V&V
methodologies.

基于 CARLA 和 ScenarioRunner 的系统级仿真验证验证框架的实例化研究，以及其在自动驾驶系统的自动生成场景方面的有效性研究。

自动化生成用于自动驾驶系统系统级仿真验证的场景

Automatic Generation of Scenarios for System-level Simulation-based  Verification of Autonomous Driving Systems

In the future, AI will increasingly find its way into systems that can
potentially cause physical harm to humans. For such safety-critical systems, it
must be demonstrated that their residual risk does not exceed what is
acceptable. This includes, in particular, the AI components that are part of
such systems' safety-related functions. Assurance cases are an intensively
discussed option today for specifying a sound and comprehensive safety argument
to demonstrate a system's safety. In previous work, it has been suggested to
argue safety for AI components by structuring assurance cases based on two
complementary risk acceptance criteria. One of these criteria is used to derive
quantitative targets regarding the AI. The argumentation structures commonly
proposed to show the achievement of such quantitative targets, however, focus
on failure rates from statistical testing. Further important aspects are only
considered in a qualitative manner -- if at all. In contrast, this paper
proposes a more holistic argumentation structure for having achieved the
target, namely a structure that integrates test results with runtime aspects
and the impact of scope compliance and test data quality in a quantitative
manner. We elaborate different argumentation options, present the underlying
mathematical considerations, and discuss resulting implications for their
practical application. Using the proposed argumentation structure might not
only increase the integrity of assurance cases but may also allow claims on
quantitative targets that would not be justifiable otherwise.

本文提出了将 AI 组件纳入保证案例的更全面的证明结构，以显示其已实现定量目标的能力，包括测试结果、运行时方面、范围符合性和测试数据质量的数量化考虑和讨论.