Large Language Models (LLMs) have demonstrated potential in cybersecurity
applications but have also caused lower confidence due to problems like
hallucinations and a lack of truthfulness. Existing benchmarks provide general
evaluations but do not sufficiently address the practical and applied aspects
of LLM performance in cybersecurity-specific tasks. To address this gap, we
introduce the SECURE (Security Extraction, Understanding \& Reasoning
Evaluation), a benchmark designed to assess LLMs performance in realistic
cybersecurity scenarios. SECURE includes six datasets focussed on the
Industrial Control System sector to evaluate knowledge extraction,
understanding, and reasoning based on industry-standard sources. Our study
evaluates seven state-of-the-art models on these tasks, providing insights into
their strengths and weaknesses in cybersecurity contexts, and offer
recommendations for improving LLMs reliability as cyber advisory tools.

大型语言模型（LLMs）在网络安全应用中显示出潜力，但也因幻觉和缺乏真实性等问题导致信心下降。现有的基准测试提供了一般评估，但不能充分解决 LLMs 在网络安全特定任务中的实际应用方面的问题。为了填补这个空白，我们引入了 SECURE（安全提取、理解与推理评估）基准测试，该基准测试旨在评估 LLMs 在现实网络安全场景中的性能。SECURE 包括六个数据集，重点关注工业控制系统领域，以评估基于行业标准来源的知识提取、理解和推理。我们的研究评估了七种最先进的模型在这些任务上的表现，为改进 LLMs 作为网络咨询工具的可靠性提供了见解和建议。

安全性：为网络安全咨询评估生成式大型语言模型

SECURE: Benchmarking Generative Large Language Models for Cybersecurity  Advisory

Globally, the external Internet is increasingly being connected to the
contemporary industrial control system. As a result, there is an immediate need
to protect the network from several threats. The key infrastructure of
industrial activity may be protected from harm by using an intrusion detection
system (IDS), a preventive measure mechanism, to recognize new kinds of
dangerous threats and hostile activities. The most recent artificial
intelligence (AI) techniques used to create IDS in many kinds of industrial
control networks are examined in this study, with a particular emphasis on
IDS-based deep transfer learning (DTL). This latter can be seen as a type of
information fusion that merge, and/or adapt knowledge from multiple domains to
enhance the performance of the target task, particularly when the labeled data
in the target domain is scarce. Publications issued after 2015 were taken into
account. These selected publications were divided into three categories:
DTL-only and IDS-only are involved in the introduction and background, and
DTL-based IDS papers are involved in the core papers of this review.
Researchers will be able to have a better grasp of the current state of DTL
approaches used in IDS in many different types of networks by reading this
review paper. Other useful information, such as the datasets used, the sort of
DTL employed, the pre-trained network, IDS techniques, the evaluation metrics
including accuracy/F-score and false alarm rate (FAR), and the improvement
gained, were also covered. The algorithms, and methods used in several studies,
or illustrate deeply and clearly the principle in any DTL-based IDS subcategory
are presented to the reader.

本文重点介绍了基于深度迁移学习的入侵检测系统 (IDS)， 特别是在工业控制网络中的应用。通过回顾相关文献，介绍了不同类型的数据集，DTL 使用的种类，预训练网络，IDS 技术以及评估指标等，经过不同研究的比较分析，证明了基于 DTL 的 IDS 可以更好地提高网络安全性。

深度迁移学习在入侵检测系统中的应用：综述

Deep Transfer Learning Applications in Intrusion Detection Systems: A  Comprehensive Review

We continue to develop our neural network (NN) based forecasting approach to
anomaly detection (AD) using the Secure Water Treatment (SWaT) industrial
control system (ICS) testbed dataset. We propose genetic algorithms (GA) to
find the best NN architecture for a given dataset, using the NAB metric to
assess the quality of different architectures. The drawbacks of the F1-metric
are analyzed. Several techniques are proposed to improve the quality of AD:
exponentially weighted smoothing, mean p-powered error measure, individual
error weight for each variable, disjoint prediction windows. Based on the
techniques used, an approach to anomaly interpretation is introduced.

利用安全水处理工业控制系统测试数据集继续发展基于神经网络的异常检测方法，并提出应用遗传算法来寻找最佳神经网络结构，使用 NAB 值评估不同结构的质量，以及引入了多种技术来改进 AD 的质量，包括指数加权平滑、均值 p - 次方误差度量、每个变量的单独误差权重和不相交预测窗口。