Validating safety-critical autonomous systems in high-dimensional domains
such as robotics presents a significant challenge. Existing black-box
approaches based on Markov chain Monte Carlo may require an enormous number of
samples, while methods based on importance sampling often rely on simple
parametric families that may struggle to represent the distribution over
failures. We propose to sample the distribution over failures using a
conditional denoising diffusion model, which has shown success in complex
high-dimensional problems such as robotic task planning. We iteratively train a
diffusion model to produce state trajectories closer to failure. We demonstrate
the effectiveness of our approach on high-dimensional robotic validation tasks,
improving sample efficiency and mode coverage compared to existing black-box
techniques.

在高维领域中验证安全关键的自主系统（如机器人）是一个重大挑战。我们提出使用条件降噪扩散模型对故障分布进行采样，并经验证在高维机器人验证任务中相比现有的黑盒技术，能够提高采样效率和模态覆盖。

基于扩散的故障采样技术对于网络物理系统

Diffusion-Based Failure Sampling for Cyber-Physical Systems

In this paper, we propose the Phy-DRL: a physics-model-regulated deep
reinforcement learning framework for safety-critical autonomous systems. The
Phy-DRL is unique in three innovations: i) proactive unknown-unknowns training,
ii) conjunctive residual control (i.e., integration of data-driven control and
physics-model-based control) and safety- \& stability-sensitive reward, and
iii) physics-model-based neural network editing, including link editing and
activation editing. Thanks to the concurrent designs, the Phy-DRL is able to 1)
tolerate unknown-unknowns disturbances, 2) guarantee mathematically provable
safety and stability, and 3) strictly comply with physical knowledge pertaining
to Bellman equation and reward. The effectiveness of the Phy-DRL is finally
validated by an inverted pendulum and a quadruped robot. The experimental
results demonstrate that compared with purely data-driven DRL, Phy-DRL features
remarkably fewer learning parameters, accelerated training and enlarged reward,
while offering enhanced model robustness and safety assurance.

本文中，我们提出了 Phy-DRL 框架：一种物理模型调节的深度强化学习框架，用于安全关键的自主系统。Phy-DRL 在三个方面独具创新：积极的未知 - 未知训练，连接残余控制，以及基于物理模型的神经网络编辑。在倒立摆和四足机器人上进行的实验结果证明了与纯数据驱动 DRL 相比，Phy-DRL 具有更少的学习参数，卓越的模型鲁棒性和安全保证。