Analysis of network traffic, anomaly detection and evaluation

With the available computer networks worldwide, it has never been easier to access, process and share information instantaneously wherever it is needed. The computer networks makes knowledge more useful and worldwide accessible at all times with the help of ever increasing processing capacities. The Internet, which is accessible all over the world and is expanding day by day, is the most important example of computer networks. The Internet has become an important infrastructure for communication, information sharing, and service providers such as governments and private institutions. The convenience that computers and Internet brought has increased the number of online Internet services and number of users who has of these services. Institutions and organizations have to deal with the tremendous amount of data they get from users to increase the quality and diversity of services. The collection and processing of data often takes place on different computer systems interconnected by computer networks. As a result, large amounts of data are required to be stored in different locations. With the Internet infrastructure, systems that process and store data remains available to users within specified limits. Besides these conveniences, computer networks and systems providing data and services are among the priority targets for those who want to harm institutions and organizations for various reasons. As even those who are not interested in cyber security are now aware that successful and destructive advanced level of cyber space attacks have resulted in massive interruptions in access to the systems that provide service, resulting in data loss, and major financial losses, as well as decreasing credibility and reputation of institutions and organizations. In addition, cyber attackers are discovering new methods every day and the detection of their activity on computer networks is becoming more and more difficult every day. There are security systems designed to work on computers and networks for the protection of computer networks and for the identification of cyber attackers' activities. Security systems, called Intrusion Detection Systems, continue to evolve in parallel with increased security concerns. In order to detect new methods discovered by cyber attackers, Intrusion Detection Systems must be designed at a level that can detect anomalies in a network. The basic approach in network traffic analysis is to detect suspicious traffic and to distinguish it from normal traffic. The Internet Engineering Task Force (IETF) established a framework to establish a common standard in the design and production of Intrusion Detection Systems. With established standards, Intrusion Detection Systems are able to share data among themselves and work together when necessary. In this context, it is possible to compare the produced systems correctly. Many criteria have been proposed for use in comparison. The most important ones are Detection Rate and False Alarm Rate. Continuous analysis of all data on the network will create an extra load on the system. Since not all the data to be analyzed have the same significance, adding a risk assessment block in addition to the existing Intrusion Detection System framework and directing the analysis block will increase the efficiency. The value of the protected data should be used as a basic input for a risk analysis. It is also necessary to determine how much of the risks can be accepted and how much can be mitigated. The crucial question here is "how to mitigate the risks". In order to secure information, Intrusion Detection Systems (IDS) should be used besides the physical and personnel safety precautions. Intrusion Detection Systems basically protect the integrity, confidentiality and accessibility of information. Technically, they use different detection approaches when doing this. The detection approaches are evaluated under 3 main topics as Statistics Based, Knowledge Based and Machine Learning Based. In the Statistical Based, the IDS follows the "normal" network activity and builds a statistical model, a pattern of it. A behavioral model is created based on the probabilities for each event by observing network traffic. And then any traffic outside the normal range is simply marked as anomaly. Because unusual events are suspicious in terms of security. Finally the Intrusion Detection System alerts for the events which are outside the model. Knowledge-based approach is widely used in Intrusion Detection Systems. In the implementation of the approach, rules are established with various methods about the protected computer system and network. Similarly, a warning is generated in violation of the rule. Artificial intelligence applications come to the forefront in Machine Learning Based Approach. Although it is an approach that produces very successful results in the analysis of highly complex data clusters, it introduces much more computational burden to computer systems. The most carefully designed part of an Intrusion Detection System framework is the block responsible for the analysis. The performance of the analysis block, where detection approaches are applied, will directly determine the performance of the Intrusion Detection System. When each detection approach is examined separately, it is seen that they have advantages over each other. But none, in all its aspects, is superior to another. It is understood that the protection capability of a system designed with a single approach will be insufficient. For this purpose, a combination of two or more methods of at least two different approaches will provide an efficient detection. A number of new detection methods have been proposed to detect the anomalies in the thesis work by looking at the footsteps of the attacks on the computer system. Knowledge Based Analysis and Statistical Based Analysis are considered together. The anomalies detected in the network traffic modeled with non-linear time series are labeled as suspicious. Today, it is very important to provide a high level of security to ensure that information is securely and reliably stored and transferred. However, secure communication over the Internet or any other network is always under threat of intruders and their activities are any set of actions that try to put the integrity, confidentiality, availability of the information at risk. Among the cyber attacks, the most common type of attack is the Denial of Service (DOS) and its variants, which aim to stop or slow down the services. In the proposed detection system, selected methods of analysis for detecting possible anomalies of early phase of DOS attacks or derivatives have been described and initial tests have been carried out with the applications. While trying to produce realistic results, it should always be known that the noise is also present in the system, which can lead to false alarms. This is very important issue that needs to be taken into consideration. Thresholds values used in the analysis block determine how far the system will perceive the noise as a real alarm, and how much the real anomalies will look normal. The IDS industry appears to be rapidly becoming a growing sector. Among the many criticisms about IDS are their performances and prices, an important component that determines the price of an IDS is how much network traffic can be captured. No matter how good a method of analysis is used, it is not possible to be certain that information security is precise unless the network traffic is thoroughly watched. IDSs are now the indispensable main element of security systems. They are rapidly evolving, and considered to be the most important technical issue in securing information in order to protect against possible attacks on computer networks and to stop as much as possible. However, most IDS produced contain commonly accepted detection methods. The novel approaches described in this thesis in terms of shedding light on the work to be done in the future offer considerable features in terms of practicality and operation. It is now possible to capture normal patterns and find anomaly ones even in mixed environments of normal and anomaly traffics.