Model-based reinforcement learning seeks to simultaneously learn the dynamics
of an unknown stochastic environment and synthesise an optimal policy for
acting in it. Ensuring the safety and robustness of sequential decisions made
through a policy in such an environment is a key challenge for policies
intended for safety-critical scenarios. In this work, we investigate two
complementary problems: first, computing reach-avoid probabilities for
iterative predictions made with dynamical models, with dynamics described by
Bayesian neural network (BNN); second, synthesising control policies that are
optimal with respect to a given reach-avoid specification (reaching a "target"
state, while avoiding a set of "unsafe" states) and a learned BNN model. Our
solution leverages interval propagation and backward recursion techniques to
compute lower bounds for the probability that a policy's sequence of actions
leads to satisfying the reach-avoid specification. Such computed lower bounds
provide safety certification for the given policy and BNN model. We then
introduce control synthesis algorithms to derive policies maximizing said lower
bounds on the safety probability. We demonstrate the effectiveness of our
method on a series of control benchmarks characterized by learned BNN dynamics
models. On our most challenging benchmark, compared to purely data-driven
policies the optimal synthesis algorithm is able to provide more than a
four-fold increase in the number of certifiable states and more than a
three-fold increase in the average guaranteed reach-avoid probability.

本研究主要探讨基于模型的强化学习中的安全性和鲁棒性问题，包括使用贝叶斯神经网络描述动态模型来计算迭代预测的到达 - 避免概率，以及使用控制综合算法综合出最佳控制策略以满足安全性约束和学习到的动态模型。