Comparision of performance of classfication methods in real data sets

Classification in data mining can be defined as creating a model with machine learning methods using observations with known output(class) values, and then using this model to predict the class values of observations with unknown class values. There are different classification algorithms that can be applied in this process and it is possible to examine the success of these algorithms on different criteria. Since the classification is a method based on prediction, it is possible to say that the most powerful criterion is the probability of the algorithm to correctly predict a class. Therefore, examining the proportion of correctly classified observations among all test observations is one of the most important criteria showing the success of classification. The main subject of this thesis is to compare the performances of 41 different classification methods existed in WEKA data mining software in classifying the real data sets and the data sets generated via simulation studies by using accuracy criterion. For this purpose, 100 real data sets with different number of observations, variables and class and 100 simulation data sets generated in different structures are used. As a result of this study, it is seen that no classifier has the best performance for each data set and different classifiers should be tested in order to achieve the best performance in all data sets. However, it is seen that Random Forest, which is a decision tree-based classification algorithm, LMT, which is an algorithm combining decision trees and logistics, and Logit Boost algorithm, which is a logistics-based classifier have the least deviation among correctly classified sample rates among all classifiers. At the same time, it was observed that the success of real data sets is superior to the success of simulation data sets.