This paper focuses on the impact of leveraging autonomous offensive
approaches in Deep Reinforcement Learning (DRL) to train more robust agents by
exploring the impact of applying adversarial learning to DRL for autonomous
security in Software Defined Networks (SDN). Two algorithms, Double Deep
Q-Networks (DDQN) and Neural Episodic Control to Deep Q-Network (NEC2DQN or
N2D), are compared. NEC2DQN was proposed in 2018 and is a new member of the
deep q-network (DQN) family of algorithms. The attacker has full observability
of the environment and access to a causative attack that uses state
manipulation in an attempt to poison the learning process. The implementation
of the attack is done under a white-box setting, in which the attacker has
access to the defender's model and experiences. Two games are played; in the
first game, DDQN is a defender and N2D is an attacker, and in second game, the
roles are reversed. The games are played twice; first, without an active
causative attack and secondly, with an active causative attack. For execution,
three sets of game results are recorded in which a single set consists of 10
game runs. The before and after results are then compared in order to see if
there was actually an improvement or degradation. The results show that with
minute parameter changes made to the algorithms, there was growth in the
attacker's role, since it is able to win games. Implementation of the
adversarial learning by the introduction of the causative attack showed the
algorithms are still able to defend the network according to their strengths.

该研究聚焦于利用自主攻击性方法在深度强化学习 (DRL) 中提升训练的鲁棒性，并探索在软件定义网络 (SDN) 中将对抗学习应用于 DRL 以实现自主安全性的影响。通过比较两个算法，DDQN 和 NEC2DQN（或 N2D），发现通过引入因果性攻击来实施对抗学习，算法仍能根据自身的具体优势保护网络。

软件定义网络中的对抗深度强化学习在网络安全中的应用

Adversarial Deep Reinforcement Learning for Cyber Security in Software  Defined Networks

Software Defined Networks have opened the door to statistical and AI-based
techniques to improve efficiency of networking. Especially to ensure a certain
Quality of Service (QoS) for specific applications by routing packets with
awareness on content nature (VoIP, video, files, etc.) and its needs (latency,
bandwidth, etc.) to use efficiently resources of a network. Predicting various
Key Performance Indicators (KPIs) at any level may handle such problems while
preserving network bandwidth. The question addressed in this work is the design
of efficient and low-cost algorithms for KPI prediction, implementable at the
local level. We focus on end-to-end latency prediction, for which we illustrate
our approaches and results on a public dataset from the recent international
challenge on GNN [1]. We propose several low complexity, locally implementable
approaches, achieving significantly lower wall time both for training and
inference, with marginally worse prediction accuracy compared to
state-of-the-art global GNN solutions.

该论文探讨了在软件定义网络中，设计低成本的本地算法来预测端到端的延迟，以满足特定应用程序的服务质量需求。研究提出了几种低复杂度的方法来预测延迟，并在公共数据集上比较了其预测和训练时间与准确性。

端到端延迟预测的低复杂度方法

Low Complexity Approaches for End-to-End Latency Prediction

Efficient network management is one of the key challenges of the constantly
growing and increasingly complex wide area networks (WAN). The paradigm shift
towards virtualized (NFV) and software defined networks (SDN) in the next
generation of mobile networks (5G), as well as the latest scientific insights
in the field of Artificial Intelligence (AI) enable the transition from
manually managed networks nowadays to fully autonomic and dynamic
self-organized networks (SON). This helps to meet the KPIs and reduce at the
same time operational costs (OPEX). In this paper, an AI driven concept is
presented for the malfunction detection in NFV applications with the help of
semi-supervised learning. For this purpose, a profile of the application under
test is created. This profile then is used as a reference to detect abnormal
behaviour. For example, if there is a bug in the updated version of the app, it
is now possible to react autonomously and roll-back the NFV app to a previous
version in order to avoid network outages.

通过利用人工智能进行半监督学习，本文介绍了一种针对 NFV 应用程序故障检测的方法，并创建了一个测试应用程序的配置文件，以检测异常行为并实现自动回滚功能，以避免网络停机。