Recent studies reveal that a well-trained deep reinforcement learning (RL)
policy can be particularly vulnerable to adversarial perturbations on input
observations. Therefore, it is crucial to train RL agents that are robust
against any attacks with a bounded budget. Existing robust training methods in
deep RL either treat correlated steps separately, ignoring the robustness of
long-term rewards, or train the agents and RL-based attacker together, doubling
the computational burden and sample complexity of the training process. In this
work, we propose a strong and efficient robust training framework for RL, named
Worst-case-aware Robust RL (WocaR-RL) that directly estimates and optimizes the
worst-case reward of a policy under bounded l_p attacks without requiring extra
samples for learning an attacker. Experiments on multiple environments show
that WocaR-RL achieves state-of-the-art performance under various strong
attacks, and obtains significantly higher training efficiency than prior
state-of-the-art robust training methods. The code of this work is available at
this https URL.

本文提出了一种强大且高效的深度强化学习鲁棒训练框架 WocaR-RL，通过直接估计和优化有界 l_p 攻击下策略的最坏情况奖励，而不需要额外的学习攻击者样本，能够在多个环境下实现最先进的性能，并获得比先前最先进的强化训练方法更高的训练效率。