Deep Reinforcement Learning (DRL) is vulnerable to small adversarial perturbations on state observations. These perturbations do not alter the environment directly but can mislead the agent into making suboptimal decisions. We analyze the Markov Decision Process (MDP) under this threat model and utilize tools from the neural net-work verification literature to enable robust train-ing for DRL under observational perturbations. Our techniques are general and can be applied to both Deep Q Networks (DQN) and Deep Deterministic Policy Gradient (DDPG) algorithms for discrete and continuous action control problems. We demonstrate that our proposed training procedure significantly improves the robustness of DQN and DDPG agents under a suite of strong white-box attacks on observations, including a few novel attacks we specifically craft. Additionally, our training procedure can produce provable certificates for the robustness of a Deep RL agent.

通过提出状态对抗马尔可夫决策过程 (SA-MDP) 的理论框架和一种新的策略正则化方法，并在多个强白盒对抗攻击中测试，我们成功地提高了一系列深层强化学习算法在离散和连续动作控制问题上的鲁棒性。

针对状态观测的鲁棒深度强化学习对抗扰动