Poisoning attacks on Reinforcement Learning (RL) systems could take advantage
of RL algorithm's vulnerabilities and cause failure of the learning. However,
prior works on poisoning RL usually either unrealistically assume the attacker
knows the underlying Markov Decision Process (MDP), or directly apply the
poisoning methods in supervised learning to RL. In this work, we build a
generic poisoning framework for online RL via a comprehensive investigation of
heterogeneous poisoning models in RL. Without any prior knowledge of the MDP,
we propose a strategic poisoning algorithm called Vulnerability-Aware
Adversarial Critic Poison (VA2C-P), which works for most policy-based deep RL
agents, closing the gap that no poisoning method exists for policy-based RL
agents. VA2C-P uses a novel metric, stability radius in RL, that measures the
vulnerability of RL algorithms. Experiments on multiple deep RL agents and
multiple environments show that our poisoning algorithm successfully prevents
agents from learning a good policy or teaches the agents to converge to a
target policy, with a limited attacking budget.

通过对异构攻击模型进行全面研究，本文建立了一个通用的强化学习在线毒化框架。在没有任何先验知识的情况下，我们提出了一种名为 VA2C-P 的策略毒化算法，通过度量 RL 的稳定半径来实现漏洞感知，成功防止了代理学习良好策略并教会代理方案收敛于目标方案.