With the growing use of information technology in all life domains, hacking
has become more negatively effective than ever before. Also with developing
technologies, attacks numbers are growing exponentially every few months and
become more sophisticated so that traditional IDS becomes inefficient detecting
them. This paper proposes a solution to detect not only new threats with higher
detection rate and lower false positive than already used IDS, but also it
could detect collective and contextual security attacks. We achieve those
results by using Networking Chatbot, a deep recurrent neural network: Long
Short Term Memory (LSTM) on top of Apache Spark Framework that has an input of
flow traffic and traffic aggregation and the output is a language of two words,
normal or abnormal. We propose merging the concepts of language processing,
contextual analysis, distributed deep learning, big data, anomaly detection of
flow analysis. We propose a model that describes the network abstract normal
behavior from a sequence of millions of packets within their context and
analyzes them in near real-time to detect point, collective and contextual
anomalies. Experiments are done on MAWI dataset, and it shows better detection
rate not only than signature IDS, but also better than traditional anomaly IDS.
The experiment shows lower false positive, higher detection rate and better
point anomalies detection. As for prove of contextual and collective anomalies
detection, we discuss our claim and the reason behind our hypothesis. But the
experiment is done on random small subsets of the dataset because of hardware
limitations, so we share experiment and our future vision thoughts as we wish
that full prove will be done in future by other interested researchers who have
better hardware infrastructure than ours.

本文介绍了一种基于深度循环神经网络技术和 Apache Spark 框架，实现了对流量异常检测的网络安全解决方案，具有更高的检测率和更低的误报率，可以检测集体和上下文安全攻击。该方法合并了语言处理、上下文分析、分布式深度学习、大数据和流量分析的概念，并在 MAWI 数据集上进行实验，结果表明，该方法不仅优于签名 IDS 和传统异常 IDS，在点异常检测方面表现更好，并有望在未来进一步证明其可靠性。