Cooperative Multi-Agent Reinforcement Learning (CMARL) strategies are well
known to be vulnerable to adversarial perturbations. Previous works on
adversarial attacks have primarily focused on white-box attacks that directly
perturb the states or actions of victim agents, often in scenarios with a
limited number of attacks. However, gaining complete access to victim agents in
real-world environments is exceedingly difficult. To create more realistic
adversarial attacks, we introduce a novel method that involves injecting
traitor agents into the CMARL system. We model this problem as a Traitor Markov
Decision Process (TMDP), where traitors cannot directly attack the victim
agents but can influence their formation or positioning through collisions. In
TMDP, traitors are trained using the same MARL algorithm as the victim agents,
with their reward function set as the negative of the victim agents' reward.
Despite this, the training efficiency for traitors remains low because it is
challenging for them to directly associate their actions with the victim
agents' rewards. To address this issue, we propose the Curiosity-Driven
Adversarial Attack (CuDA2) framework. CuDA2 enhances the efficiency and
aggressiveness of attacks on the specified victim agents' policies while
maintaining the optimal policy invariance of the traitors. Specifically, we
employ a pre-trained Random Network Distillation (RND) module, where the extra
reward generated by the RND module encourages traitors to explore states
unencountered by the victim agents. Extensive experiments on various scenarios
from SMAC demonstrate that our CuDA2 framework offers comparable or superior
adversarial attack capabilities compared to other baselines.

对于合作多智能体强化学习（CMARL）策略容易受到对抗性干扰的问题，研究者们引入了一种名为 Traitor Markov Decision Process（TMDP）的模型作为现实场景下更真实的对抗性攻击方法。为了提高攻击效率，研究者们还提出了一种名为 Curiosity-Driven Adversarial Attack（CuDA2）的框架，其中 Random Network Distillation（RND）模块对攻击者提供额外的奖励以激励其探索未被受害者智能体所遇到过的状态。实验结果表明，CuDA2 框架在不同场景下的对抗性攻击能力相当或优于其他基准方法。