For machine learning components used as part of autonomous systems (AS) in
carrying out critical tasks it is crucial that assurance of the models can be
maintained in the face of post-deployment changes (such as changes in the
operating environment of the system). A critical part of this is to be able to
monitor when the performance of the model at runtime (as a result of changes)
poses a safety risk to the system. This is a particularly difficult challenge
when ground truth is unavailable at runtime. In this paper we introduce a
process for creating safety monitors for ML components through the use of
degraded datasets and machine learning. The safety monitor that is created is
deployed to the AS in parallel to the ML component to provide a prediction of
the safety risk associated with the model output. We demonstrate the viability
of our approach through some initial experiments using publicly available speed
sign datasets.

通过使用退化数据集和机器学习，本文介绍了一个用于为机器学习组件创建安全监视器的过程，该安全监视器与自主系统并行部署，以提供模型输出的安全风险预测。我们通过使用公开可用的速度标志数据集进行一些初步实验，证明了我们方法的可行性。

学习机器学习组件的运行时安全监控

Learning Run-time Safety Monitors for Machine Learning Components

Many organizations are developing autonomous driving systems, which are
expected to be deployed at a large scale in the near future. Despite this,
there is a lack of agreement on appropriate methods to test, debug, and certify
the performance of these systems. One of the main challenges is that many
autonomous driving systems have machine learning components, such as deep
neural networks, for which formal properties are difficult to characterize. We
present a testing framework that is compatible with test case generation and
automatic falsification methods, which are used to evaluate cyber-physical
systems. We demonstrate how the framework can be used to evaluate closed-loop
properties of an autonomous driving system model that includes the ML
components, all within a virtual environment. We demonstrate how to use test
case generation methods, such as covering arrays, as well as requirement
falsification methods to automatically identify problematic test scenarios. The
resulting framework can be used to increase the reliability of autonomous
driving systems.

本文针对包含机器学习组件的自动驾驶系统，提出了一个测试框架，利用测试用例生成和自动实现方法进行评估，并演示了如何使用覆盖数组等测试用例生成方法以及需求实现方法来自动识别问题情景，从而提高自动驾驶系统的可靠性。