Securing networked infrastructures is important in the real world. The problem of deploying security resources to protect against an attacker in networked domains can be modeled as Network Security Games (NSGs). Unfortunately, existing approaches, including the deep learning-based approaches, are inefficient to solve large-scale extensive-form NSGs. In this paper, we propose a novel learning paradigm, NSG-NFSP, to solve large-scale extensive-form NSGs based on Neural Fictitious Self-Play (NFSP). Our main contributions include: i) reforming the best response (BR) policy network in NFSP to be a mapping from action-state pair to action-value, to make the calculation of BR possible in NSGs; ii) converting the average policy network of an NFSP agent into a metric-based classifier, helping the agent to assign distributions only on legal actions rather than all actions; iii) enabling NFSP with high-level actions, which can benefit training efficiency and stability in NSGs; and iv) leveraging information contained in graphs of NSGs by learning efficient graph node embeddings. Our algorithm significantly outperforms state-of-the-art algorithms in both scalability and solution quality.

本文提出了一种基于神经虚构自我博弈的学习方法，NSG-NFSP，来解决大规模复杂形式网络安全博弈，主要贡献包括改进最佳响应策略网络，转换平均策略网络为基于度量的分类器，加入高层次动作，以及利用 NSG 图节点嵌入信息进行学习。该算法在可伸缩性和解决方案质量方面明显优于现有的算法。

通过神经虚幻自我对抗解决大规模的延拓形网络安全博弈