In recent years, there has been a growing interest in using Machine Learning
(ML), especially Deep Learning (DL) to solve Network Intrusion Detection (NID)
problems. However, the feature distribution shift problem remains a difficulty,
because the change in features' distributions over time negatively impacts the
model's performance. As one promising solution, model pretraining has emerged
as a novel training paradigm, which brings robustness against feature
distribution shift and has proven to be successful in Computer Vision (CV) and
Natural Language Processing (NLP). To verify whether this paradigm is
beneficial for NID problem, we propose SwapCon, a ML model in the context of
NID, which compresses shift-invariant feature information during the
pretraining stage and refines during the finetuning stage. We exemplify the
evidence of feature distribution shift using the Kyoto2006+ dataset. We
demonstrate how pretraining a model with the proper size can increase
robustness against feature distribution shifts by over 8%. Moreover, we show
how an adequate numerical embedding strategy also enhances the performance of
pretrained models. Further experiments show that the proposed SwapCon model
also outperforms eXtreme Gradient Boosting (XGBoost) and K-Nearest Neighbor
(KNN) based models by a large margin.

基于 SwapCon 模型，我们提出了一种在预训练阶段压缩移不变特征信息并在微调阶段精细调整的机器学习模型，证明这种预训练范式在网络入侵检测问题中可以提高对特征分布变化的鲁棒性超过 8%，并且比基于 eXtreme Gradient Boosting (XGBoost) 和 K - 最近邻 (KNN) 的模型表现更好。

特征分布偏移消减的对比预训练方法用于入侵检测

Feature Distribution Shift Mitigation with Contrastive Pretraining for  Intrusion Detection

It has become a popular paradigm to transfer the knowledge of large-scale
pre-trained models to various downstream tasks via fine-tuning the entire model
parameters. However, with the growth of model scale and the rising number of
downstream tasks, this paradigm inevitably meets the challenges in terms of
computation consumption and memory footprint issues. Recently,
Parameter-Efficient Fine-Tuning (PEFT) (e.g., Adapter, LoRA, BitFit) shows a
promising paradigm to alleviate these concerns by updating only a portion of
parameters. Despite these PEFTs having demonstrated satisfactory performance in
natural language processing, it remains under-explored for the question of
whether these techniques could be transferred to graph-based tasks with Graph
Transformer Networks (GTNs). Therefore, in this paper, we fill this gap by
providing extensive benchmarks with traditional PEFTs on a range of graph-based
downstream tasks. Our empirical study shows that it is sub-optimal to directly
transfer existing PEFTs to graph-based tasks due to the issue of feature
distribution shift. To address this issue, we propose a novel structure-aware
PEFT approach, named G-Adapter, which leverages graph convolution operation to
introduce graph structure (e.g., graph adjacent matrix) as an inductive bias to
guide the updating process. Besides, we propose Bregman proximal point
optimization to further alleviate feature distribution shift by preventing the
model from aggressive update. Extensive experiments demonstrate that G-Adapter
obtains the state-of-the-art performance compared to the counterparts on nine
graph benchmark datasets based on two pre-trained GTNs, and delivers tremendous
memory footprint efficiency compared to the conventional paradigm.

本文旨在探索将 Parameter-Efficient Fine-Tuning (PEFT) 技术应用于基于图的任务，提出了一种名为 G-Adapter 的新型结构感知 PEFT 方法，并使用 Bregman Proximal Point 方法进一步缓解特征分布变化问题，研究结果表明，G-Adapter 相对于现有的 PEFT 方法，在 9 个基于图的基准数据集上获得了最先进的性能，并且相比于传统的方法提供了巨大的内存节省。