Machine Learning has been steadily gaining traction for its use in Anomaly-based Network Intrusion Detection Systems (A-NIDS). Research into this domain is frequently performed using the KDD~CUP~99 dataset as a benchmark. Several studies question its usability while constructing a contemporary NIDS, due to the skewed response distribution, non-stationarity, and failure to incorporate modern attacks. In this paper, we compare the performance for KDD-99 alternatives when trained using classification models commonly found in literature: Neural Network, Support Vector Machine, Decision Tree, Random Forest, Naive Bayes and K-Means. Applying the SMOTE oversampling technique and random undersampling, we create a balanced version of NSL-KDD and prove that skewed target classes in KDD-99 and NSL-KDD hamper the efficacy of classifiers on minority classes (U2R and R2L), leading to possible security risks. We explore UNSW-NB15, a modern substitute to KDD-99 with greater uniformity of pattern distribution. We benchmark this dataset before and after SMOTE oversampling to observe the effect on minority performance. Our results indicate that classifiers trained on UNSW-NB15 match or better the Weighted F1-Score of those trained on NSL-KDD and KDD-99 in the binary case, thus advocating UNSW-NB15 as a modern substitute to these datasets.

本文研究了机器学习在基于异常的网络入侵检测系统(A-NIDS)中的应用，并发现使用KDD-99数据集进行研究存在偏差。通过比较许多分类模型的性能，以及使用SMOTE过采样技术和随机欠采样技术来创建一个平衡的NSL-KDD数据集，我们证明了数据偏斜会影响分类模型的准确性，从而导致潜在的安全风险。我们探索了一个现代的替代品UNSW-NB15，并发现在二元情况下，使用SMOTE进行训练后的UNSW-NB15数据集可以与NSL-KDD和KDD-99相匹配或超过加权F1分数，因此值得被使用作为现代替代品。

基于异常的网络入侵检测数据集基准测试：KDD CUP 99替代方案