Cooperative multi-agent reinforcement learning (cMARL) has many real
applications, but the policy trained by existing cMARL algorithms is not robust
enough when deployed. There exist also many methods about adversarial attacks
on the RL system, which implies that the RL system can suffer from adversarial
attacks, but most of them focused on single agent RL. In this paper, we propose
a \textit{sparse adversarial attack} on cMARL systems. We use (MA)RL with
regularization to train the attack policy. Our experiments show that the policy
trained by the current cMARL algorithm can obtain poor performance when only
one or a few agents in the team (e.g., 1 of 8 or 5 of 25) were attacked at a
few timesteps (e.g., attack 3 of total 40 timesteps).

本篇论文介绍了 cMARL 系统中的稀疏对抗攻击方法，并探究了该方法的训练策略与效果。实验结果表明，该方法能够显著降低 cMARL 系统中受到少量攻击智能体的性能。

多智能体强化学习中的稀疏对抗攻击

Sparse Adversarial Attack in Multi-agent Reinforcement Learning

Modern deep neural networks(DNNs) are vulnerable to adversarial samples.
Sparse adversarial samples are a special branch of adversarial samples that can
fool the target model by only perturbing a few pixels. The existence of the
sparse adversarial attack points out that DNNs are much more vulnerable than
people believed, which is also a new aspect for analyzing DNNs. However,
current sparse adversarial attack methods still have some shortcomings on both
sparsity and invisibility. In this paper, we propose a novel two-stage
distortion-aware greedy-based method dubbed as "GreedyFool". Specifically, it
first selects the most effective candidate positions to modify by considering
both the gradient(for adversary) and the distortion map(for invisibility), then
drops some less important points in the reduce stage. Experiments demonstrate
that compared with the start-of-the-art method, we only need to modify
$3\times$ fewer pixels under the same sparse perturbation setting. For target
attack, the success rate of our method is 9.96\% higher than the
start-of-the-art method under the same pixel budget. Code can be found at
this https URL

我们提出了一种新的基于贪婪算法的两阶段畸变感知方法 GreedyFool，实验结果表明，相比于现有方法，我们只需要在相同的稀疏扰动设置下修改 3 倍少的像素，对于目标攻击而言，我们的方法比现有方法在相同像素预算下成功率高出 9.96%。