Network anomaly detection using linear correlation based feature selection and simplified mahalanobis distance

Network security plays a crucial role in protecting the system and taking precautions against network attacks and security breaches. The issue of security has gained even more importance with the emergence of new technologies such as the Internet of Things (IoT). Intrusion detection systems are widely used in security research and classified into two categories such as machine learning and statistical-based techniques. Many techniques have been developed based on signature and profile to detect network breaches. The main obstacle of detecting anomalies in intrusion detection systems is the high dimensionality and redundant features of the dataset, which increases the complexity of the systems and reduces the detection rate of anomalies. This increase in complexity requires new and different techniques to detect and prevent network attacks. The purpose of this study is to apply the simplified Mahalanobis distance (SMD) instead of the traditional Mahalanobis distance, which requires less computational complexity, and to investigate the effectiveness of statistical-based methods for anomaly detection. The experiments were applied using an approach consolidating linear correlation-based feature selection (LCFS) and SMD techniques at the big data scale using the UNSW-NB15 dataset, which includes 2.5 million rows. The findings showed that our model obtained the lowest false alarm rate between 0.88% and 1.04% and the highest detection rate of 99.94%, which are significant for intrusion detection systems. Furthermore, it showed that our approach could obtain results close to each other compared to literature studies based on machine learning models. Consequently, this study proposes an anomaly-based approach that combines LCFS and SMD for IoT environments in terms of literature contribution.