This paper presents the FlowTransformer framework, a novel approach for
implementing transformer-based Network Intrusion Detection Systems (NIDSs).
FlowTransformer leverages the strengths of transformer models in identifying
the long-term behaviour and characteristics of networks, which are often
overlooked by most existing NIDSs. By capturing these complex patterns in
network traffic, FlowTransformer offers a flexible and efficient tool for
researchers and practitioners in the cybersecurity community who are seeking to
implement NIDSs using transformer-based models. FlowTransformer allows the
direct substitution of various transformer components, including the input
encoding, transformer, classification head, and the evaluation of these across
any flow-based network dataset. To demonstrate the effectiveness and efficiency
of the FlowTransformer framework, we utilise it to provide an extensive
evaluation of various common transformer architectures, such as GPT 2.0 and
BERT, on three commonly used public NIDS benchmark datasets. We provide results
for accuracy, model size and speed. A key finding of our evaluation is that the
choice of classification head has the most significant impact on the model
performance. Surprisingly, Global Average Pooling, which is commonly used in
text classification, performs very poorly in the context of NIDS. In addition,
we show that model size can be reduced by over 50\%, and inference and training
times improved, with no loss of accuracy, by making specific choices of input
encoding and classification head instead of other commonly used alternatives.

本文提出了 FlowTransformer 框架，一种新的基于 transformer 的网络入侵检测系统（NIDS）实现方法，FlowTransformer 利用 transformer 模型在识别网络的长期行为和特征方面的优势，通过捕捉网络流量中的复杂模式，为在网络安全领域中寻求使用 transformer-based 模型实现 NIDS 的研究人员和从业人员提供了灵活高效的工具。