For machine learning components used as part of autonomous systems (AS) in
carrying out critical tasks it is crucial that assurance of the models can be
maintained in the face of post-deployment changes (such as changes in the
operating environment of the system). A critical part of this is to be able to
monitor when the performance of the model at runtime (as a result of changes)
poses a safety risk to the system. This is a particularly difficult challenge
when ground truth is unavailable at runtime. In this paper we introduce a
process for creating safety monitors for ML components through the use of
degraded datasets and machine learning. The safety monitor that is created is
deployed to the AS in parallel to the ML component to provide a prediction of
the safety risk associated with the model output. We demonstrate the viability
of our approach through some initial experiments using publicly available speed
sign datasets.

通过使用退化数据集和机器学习，本文介绍了一个用于为机器学习组件创建安全监视器的过程，该安全监视器与自主系统并行部署，以提供模型输出的安全风险预测。我们通过使用公开可用的速度标志数据集进行一些初步实验，证明了我们方法的可行性。

学习机器学习组件的运行时安全监控

Learning Run-time Safety Monitors for Machine Learning Components

As Autonomous Systems (AS) become more ubiquitous in society, more
responsible for our safety and our interaction with them more frequent, it is
essential that they are trustworthy. Assessing the trustworthiness of AS is a
mandatory challenge for the verification and development community. This will
require appropriate standards and suitable metrics that may serve to
objectively and comparatively judge trustworthiness of AS across the broad
range of current and future applications. The meta-expression `trustworthiness'
is examined in the context of AS capturing the relevant qualities that comprise
this term in the literature. Recent developments in standards and frameworks
that support assurance of autonomous systems are reviewed. A list of key
challenges are identified for the community and we present an outline of a
process that can be used as a trustworthiness assessment framework for AS.

研究探讨了自主系统的信任度和相关标准，并提出了一个可用于自主系统的信任度评估框架的大纲。

评估自主系统的可信度

Assessing Trustworthiness of Autonomous Systems

The application of machine learning (ML) in computer systems introduces not
only many benefits but also risks to society. In this paper, we develop the
concept of ML governance to balance such benefits and risks, with the aim of
achieving responsible applications of ML. Our approach first systematizes
research towards ascertaining ownership of data and models, thus fostering a
notion of identity specific to ML systems. Building on this foundation, we use
identities to hold principals accountable for failures of ML systems through
both attribution and auditing. To increase trust in ML systems, we then survey
techniques for developing assurance, i.e., confidence that the system meets its
security requirements and does not exhibit certain known failures. This leads
us to highlight the need for techniques that allow a model owner to manage the
life cycle of their system, e.g., to patch or retire their ML system. Put
altogether, our systematization of knowledge standardizes the interactions
between principals involved in the deployment of ML throughout its life cycle.
We highlight opportunities for future work, e.g., to formalize the resulting
game between ML principals.

本文提出了机器学习治理的概念，并使用身份标识来确保机器学习系统的所有者对于系统的失败负责，从而增强人们对机器学习系统的信任，为未来的工作提供了机会。