Reinforcement Learning (RL) has shown remarkable success in solving
relatively complex tasks, yet the deployment of RL systems in real-world
scenarios poses significant challenges related to safety and robustness. This
paper aims to identify and further understand those challenges thorough the
exploration of the main dimensions of the safe and robust RL landscape,
encompassing algorithmic, ethical, and practical considerations. We conduct a
comprehensive review of methodologies and open problems that summarizes the
efforts in recent years to address the inherent risks associated with RL
applications.
After discussing and proposing definitions for both safe and robust RL, the
paper categorizes existing research works into different algorithmic approaches
that enhance the safety and robustness of RL agents. We examine techniques such
as uncertainty estimation, optimisation methodologies, exploration-exploitation
trade-offs, and adversarial training. Environmental factors, including
sim-to-real transfer and domain adaptation, are also scrutinized to understand
how RL systems can adapt to diverse and dynamic surroundings. Moreover, human
involvement is an integral ingredient of the analysis, acknowledging the broad
set of roles that humans can take in this context.
Importantly, to aid practitioners in navigating the complexities of safe and
robust RL implementation, this paper introduces a practical checklist derived
from the synthesized literature. The checklist encompasses critical aspects of
algorithm design, training environment considerations, and ethical guidelines.
It will serve as a resource for developers and policymakers alike to ensure the
responsible deployment of RL systems in many application domains.

通过综述方法与开放问题对最近几年来 RL 的安全和稳健性的相关研究工作进行总结，本文主要关注 RL 系统在现实场景中的安全性和稳健性挑战，探讨了算法、伦理和实践考虑等方面的主要维度以及如何增强 RL 代理的安全性和稳健性，同时讨论了环境因素和人的参与等影响因素，最后提出了一个实用的检查清单，以帮助从业者在各个应用领域负责任地部署 RL 系统。