Communication networks able to withstand hostile environments are critically important for disaster relief operations. In this paper, we consider a challenging scenario where drones have been compromised in the supply chain, during their manufacture, and harbour malicious software capable of wide-ranging and infectious disruption. We investigate multi-agent deep reinforcement learning as a tool for learning defensive strategies that maximise communications bandwidth despite continual adversarial interference. Using a public challenge for learning network resilience strategies, we propose a state-of-the-art expert technique and study its superiority over deep reinforcement learning agents. Correspondingly, we identify three specific methods for improving the performance of our learning-based agents: (1) ensuring each observation contains the necessary information, (2) using expert agents to provide a curriculum for learning, and (3) paying close attention to reward. We apply our methods and present a new mixed strategy enabling expert and learning-based agents to work together and improve on all prior results.

在本研究中，我们考虑了一种具有挑战性的情境，其中无人机在供应链的制造过程中受到了损害，并携带着能够广泛传播和带来破坏的恶意软件。我们通过研究多智能体深度强化学习作为一种学习防御策略、最大化通信带宽的工具，来解决这一问题。通过使用一个公共挑战任务来学习网络韧性策略，我们提出了一种先进的专家技术，并研究了其相对于深度强化学习智能体的优越性。相应地，我们确定了三种改进学习智能体性能的具体方法：（1）保证每个观测包含必要的信息，（2）使用专家智能体为学习提供课程，（3）密切关注奖励。我们应用了这些方法，并提出了一种新的混合策略，使专家智能体和学习智能体能够共同合作，并在所有先前结果的基础上取得改进。

金丝雀与哨声：具有或不具有深度强化学习的弹性无人机通信网络