Cyber-physical systems (CPS) with reinforcement learning (RL)-based
controllers are increasingly being deployed in complex physical environments
such as autonomous vehicles, the Internet-of-Things(IoT), and smart cities. An
important property of a CPS is tolerance; i.e., its ability to function safely
under possible disturbances and uncertainties in the actual operation. In this
paper, we introduce a new, expressive notion of tolerance that describes how
well a controller is capable of satisfying a desired system requirement,
specified using Signal Temporal Logic (STL), under possible deviations in the
system. Based on this definition, we propose a novel analysis problem, called
the tolerance falsification problem, which involves finding small deviations
that result in a violation of the given requirement. We present a novel,
two-layer simulation-based analysis framework and a novel search heuristic for
finding small tolerance violations. To evaluate our approach, we construct a
set of benchmark problems where system parameters can be configured to
represent different types of uncertainties and disturbancesin the system. Our
evaluation shows that our falsification approach and heuristic can effectively
find small tolerance violations.

在该研究论文中，介绍了一种新的容错概念，旨在描述控制器在系统中可能出现的偏差下满足特定系统要求的能力。基于该定义，提出了一种新颖的分析问题 —— 容错验证问题，其中包括寻找导致给定要求违反的小偏差。通过构建一系列基准问题来评估该方法，在系统中可以配置系统参数以表示不同类型的不确定性和干扰，实验表明，我们的验证方法和启发式算法能够有效地找到小偏差。

强化学习控制器对网络物理系统中的偏差容忍性

Tolerance of Reinforcement Learning Controllers against Deviations in  Cyber Physical Systems

We formulate intrusion tolerance for a system with service replicas as a
two-level optimal control problem. On the local level node controllers perform
intrusion recovery, and on the global level a system controller manages the
replication factor. The local and global control problems can be formulated as
classical problems in operations research, namely, the machine replacement
problem and the inventory replenishment problem. Based on this formulation, we
design TOLERANCE, a novel control architecture for intrusion-tolerant systems.
We prove that the optimal control strategies on both levels have threshold
structure and design efficient algorithms for computing them. We implement and
evaluate TOLERANCE in an emulation environment where we run 10 types of network
intrusions. The results show that TOLERANCE can improve service availability
and reduce operational cost compared with state-of-the-art intrusion-tolerant
systems.

我们设计了一个新颖的控制架构 TOLERANCE 来应对具有服务副本的系统中的入侵容忍问题。我们证明了在两个控制层级上的最优控制策略具有阈值结构，并设计了有效的算法来计算它们。通过在仿真环境中运行 10 种网络入侵，我们验证了 TOLERANCE 可以提高服务可用性并降低运营成本。

通过两级反馈控制实现网络系统的入侵容忍

Intrusion Tolerance for Networked Systems through Two-Level Feedback  Control

Approximate Bayesian computation (ABC) is a likelihood-free approach for
Bayesian inferences based on a rejection algorithm method that applies a
tolerance of dissimilarity between summary statistics from observed and
simulated data. Although several improvements to the algorithm have been
proposed, none of these improvements avoid the following two sources of
approximation: 1) lack of sufficient statistics: sampling is not from the true
posterior density given data but from an approximate posterior density given
summary statistics; and 2) non-zero tolerance: sampling from the posterior
density given summary statistics is achieved only in the limit of zero
tolerance. The first source of approximation can be improved by adding a
summary statistic, but an increase in the number of summary statistics could
introduce additional variance caused by the low acceptance rate. Consequently,
many researchers have attempted to develop techniques to choose informative
summary statistics. The present study evaluated the utility of a kernel-based
ABC method (Fukumizu et al. 2010, arXiv:1009.5736 and 2011, NIPS 24: 1549-1557)
for complex problems that demand many summary statistics. Specifically, kernel
ABC was applied to population genetic inference. We demonstrate that, in
contrast to conventional ABCs, kernel ABC can incorporate a large number of
summary statistics while maintaining high performance of the inference.

本文介绍了一种基于 kernel 的 ABC 方法，它能够应对需要使用多个总结统计量的复杂问题，并表现出高效的推理性能。