The safety of decentralized reinforcement learning (RL) is a challenging problem since malicious agents can share their poisoned policies with benign agents. The paper investigates a cooperative backdoor attack in a decentralized reinforcement learning scenario. Differing from the existing methods that hide a whole backdoor attack behind their shared policies, our method decomposes the backdoor behavior into multiple components according to the state space of RL. Each malicious agent hides one component in its policy and shares its policy with the benign agents. When a benign agent learns all the poisoned policies, the backdoor attack is assembled in its policy. The theoretical proof is given to show that our cooperative method can successfully inject the backdoor into the RL policies of benign agents. Compared with the existing backdoor attacks, our cooperative method is more covert since the policy from each attacker only contains a component of the backdoor attack and is harder to detect. Extensive simulations are conducted based on Atari environments to demonstrate the efficiency and covertness of our method. To the best of our knowledge, this is the first paper presenting a provable cooperative backdoor attack in decentralized reinforcement learning.

该研究论文调查了一种在分散式增强学习场景中的合作后门攻击方法，通过将后门行为分解为多个组件并将其隐藏在恶意代理的策略中，并与良性代理共享策略，成功地向良性代理的增强学习策略中注入了后门攻击。与现有的后门攻击相比，该合作方法更加隐蔽，因为每个攻击者的策略只包含后门攻击的一个组件，难以检测。通过基于Atari环境的广泛模拟实验证明了该方法的效率和隐蔽性。据我们所知，这是首次提出在分散式增强学习中的可证明的合作后门攻击的论文。

分布式强化学习中的合作后门攻击与理论保证