The rapid increase in the number of cyber-attacks in recent years raises the
need for principled methods for defending networks against malicious actors.
Deep reinforcement learning (DRL) has emerged as a promising approach for
mitigating these attacks. However, while DRL has shown much potential for
cyber-defence, numerous challenges must be overcome before DRL can be applied
to autonomous cyber-operations (ACO) at scale. Principled methods are required
for environments that confront learners with very high-dimensional state
spaces, large multi-discrete action spaces, and adversarial learning. Recent
works have reported success in solving these problems individually. There have
also been impressive engineering efforts towards solving all three for
real-time strategy games. However, applying DRL to the full ACO problem remains
an open challenge. Here, we survey the relevant DRL literature and
conceptualize an idealised ACO-DRL agent. We provide: i.) A summary of the
domain properties that define the ACO problem; ii.) A comprehensive evaluation
of the extent to which domains used for benchmarking DRL approaches are
comparable to ACO; iii.) An overview of state-of-the-art approaches for scaling
DRL to domains that confront learners with the curse of dimensionality, and;
iv.) A survey and critique of current methods for limiting the exploitability
of agents within adversarial settings from the perspective of ACO. We conclude
with open research questions that we hope will motivate future directions for
researchers and practitioners working on ACO.

近年来网络攻击数量的快速增加使得有必要提出原则性方法来防御恶意行为者。深度强化学习已成为一种有望减轻这些攻击的方法，但在大规模应用于自主网络操作之前，需要克服许多挑战。本文调研了相关的深度强化学习文献，并构思了一个理想的自主网络操作的深度强化学习代理，提供了领域特性概要、深度强化学习方法与自主网络操作之间的比较、扩展深度强化学习的方法以应对高维状态空间挑战以及限制对抗环境中代理利用性的现有方法的概述和评价，并提出未来的研究问题。