Design of an adaptive intrusion detection and prevention system for software-defined networks

The rapid increase in network traffic has recently led to the importance of flow-based intrusion detection systems processing a small amount of traffic data Furthermore, anomaly-based methods, which can identify unknown attacks are also integrated into these systems. In this study, the focus is concentrated on the detection of anomalous network traffic (or intrusions) from flow-based data using unsupervised deep learning methods with semi-supervised learning approach. More specifically, Autoencoder and Variational Autoencoder methods were employed to identify unknown attacks using flow features. In the experiments carried out, the flow-based features extracted out of network traffic data, including typical and different types of attacks, were used. The Receiver Operating Characteristics (ROC) and the area under ROC curve, resulting from these methods were calculated and compared with One-Class Support Vector Machine. The ROC curves were examined in detail to analyze the performance of the methods in various threshold values. The experimental results show that Variational Autoencoder performs, for the most part, better than Autoencoder and One-Class Support Vector Machine. In the second part of the study, SAnDet architecture, which can do an anomaly-based intrusion detection by taking advantage of the capabilities offered by SDN architecture, is presented and implemented in Floodlight controller. A detailed description of this system which consists of three main modules which are statistics collector, anomaly detector and anomaly prevention is given. More specifically, RNN which is a special variant of the autoencoder, and EncDecAD methods which a special type of LSTM networks that can produce successful results especially in given data series, were used to identify unknown attacks using flow features collected from OpenFlow switches. In experiments, flow-based features extracted from network traffic data including different types of attacks, are given as input into models as time series. The results of the methods are calculated according to the ROC and AUC metrics. ROC curves have been examined in detail to analyze the performance of methods at various threshold values. Experimental results show that EncDecAD outperforms RNN and methods proposed in the literature.