This paper presents a safe learning framework that employs an adaptive model
learning algorithm together with barrier certificates for systems with possibly
nonstationary agent dynamics. To extract the dynamic structure of the model, we
use a sparse optimization technique. We use the learned model in combination
with control barrier certificates which constrain policies (feedback
controllers) in order to maintain safety, which refers to avoiding particular
undesirable regions of the state space. Under certain conditions, recovery of
safety in the sense of Lyapunov stability after violations of safety due to the
nonstationarity is guaranteed. In addition, we reformulate an action-value
function approximation to make any kernel-based nonlinear function estimation
method applicable to our adaptive learning framework. Lastly, solutions to the
barrier-certified policy optimization are guaranteed to be globally optimal,
ensuring the greedy policy improvement under mild conditions. The resulting
framework is validated via simulations of a quadrotor, which has previously
been used under stationarity assumptions in the safe learnings literature, and
is then tested on a real robot, the brushbot, whose dynamics is unknown, highly
complex and nonstationary.

本文提出了一个安全学习框架，该框架采用自适应模型学习算法以及障碍证书，用于具有可能非平稳智能体动态的系统。本文使用稀疏优化技术提取模型的动态结构，并结合控制障碍证书来保持安全。在一定条件下，保证了违反安全性后的 Lypunov 稳定恢复。最终证明了该框架通过仿真和测试的方式，在具有未知、高度复杂和非平稳动态的机器人系统中是有效的。