We implemented and evaluated an automated cyber defense agent. The agent
takes security alerts as input and uses reinforcement learning to learn a
policy for executing predefined defensive measures. The defender policies were
trained in an environment intended to simulate a cyber attack. In the
simulation, an attacking agent attempts to capture targets in the environment,
while the defender attempts to protect them by enabling defenses. The
environment was modeled using attack graphs based on the Meta Attack Language
language. We assumed that defensive measures have downtime costs, meaning that
the defender agent was penalized for using them. We also assumed that the
environment was equipped with an imperfect intrusion detection system that
occasionally produces erroneous alerts based on the environment state. To
evaluate the setup, we trained the defensive agent with different volumes of
intrusion detection system noise. We also trained agents with different
attacker strategies and graph sizes. In experiments, the defensive agent using
policies trained with reinforcement learning outperformed agents using
heuristic policies. Experiments also demonstrated that the policies could
generalize across different attacker strategies. However, the performance of
the learned policies decreased as the attack graphs increased in size.

实现并评估了一种基于强化学习的自动化网络防御代理程序，该代理以安全警报作为输入并使用强化学习学习执行预定义的防御措施的策略，使用攻击图模拟网络攻击的环境中，被攻击者执行保护任务。该方法通过使用不同大小的攻击图、攻击者攻击策略和不同的检测系统噪声来进行评估。实验表明，通过强化学习训练的防御代理程序的性能优于使用启发式策略的代理程序，并能够概括不同的攻击者策略。

使用基于图的网络攻击模拟训练自动化防御策略

Training Automated Defense Strategies Using Graph-based Cyber Attack  Simulations

Attack graphs (AG) are used to assess pathways availed by cyber adversaries
to penetrate a network. State-of-the-art approaches for AG generation focus
mostly on deriving dependencies between system vulnerabilities based on network
scans and expert knowledge. In real-world operations however, it is costly and
ineffective to rely on constant vulnerability scanning and expert-crafted AGs.
We propose to automatically learn AGs based on actions observed through
intrusion alerts, without prior expert knowledge. Specifically, we develop an
unsupervised sequence learning system, SAGE, that leverages the temporal and
probabilistic dependence between alerts in a suffix-based probabilistic
deterministic finite automaton (S-PDFA) -- a model that accentuates infrequent
severe alerts and summarizes paths leading to them. AGs are then derived from
the S-PDFA on a per-objective, per-victim basis. Tested with intrusion alerts
collected through Collegiate Penetration Testing Competition, SAGE compresses
over 330k alerts into 93 AGs. These AGs reflect the strategies used by the
participating teams. The AGs are succinct, interpretable, and capture
behavioral dynamics, e.g., that attackers will often follow shorter paths to
re-exploit objectives.

通过利用未经专家知识的入侵警报自动学习攻击图，SAGE 可在特定目标和受害者的基础上获得路径，以便有效地评估网络安全威胁。

基于入侵警报的攻击图提取器 SAGE

SAGE: Intrusion Alert-driven Attack Graph Extractor

Assessing network security is a complex and difficult task. Attack graphs
have been proposed as a tool to help network administrators understand the
potential weaknesses of their network. However, a problem has not yet been
addressed by previous work on this subject; namely, how to actually execute and
validate the attack paths resulting from the analysis of the attack graph. In
this paper we present a complete PDDL representation of an attack model, and an
implementation that integrates a planner into a penetration testing tool. This
allows to automatically generate attack paths for penetration testing
scenarios, and to validate these attacks by executing the corresponding actions
-including exploits- against the real target network. We present an algorithm
for transforming the information present in the penetration testing tool to the
planning domain, and show how the scalability issues of attack graphs can be
solved using current planners. We include an analysis of the performance of our
solution, showing how our model scales to medium-sized networks and the number
of actions available in current penetration testing tools.

本文介绍了一个完整的 PDDL 攻击模型表示，整合规划器到渗透测试工具中的实现，可以自动为渗透测试方案生成攻击路径，并通过对真实目标网络执行相应的操作来验证这些攻击，包括利用漏洞等。同时，作者提出一种从渗透测试工具中提取信息到规划领域的算法，并展示如何使用当前的规划器解决攻击图的可伸缩性问题。作者的解决方案的性能分析表明，该模型可扩展到中等规模网络和当前渗透测试工具中可用的操作数量。