A Comparative Study of Statistical Models for Feature Selection Methods in Text Categorization

In consequence of change and developments in the world of technology, the data have been started to transfer to the digital environment rapidly and categorization task of digital documents has become difficult and complicated. Therefore, researchers have focused on doing more research in field of machine learning to provide a more effective solution in terms of resources and time. A major problem of text categorization is the high dimension of the feature space. Feature selection methods are widely used for choosing the subset of features in last decades. In order to maximize the text classification efficiency, some machine learning algorithms and feature selection methods are studied in a comparative way. The experiments are conducted with Reuters-21578 "ApteMod" version, The 4-Universities and 20- Newsgroups "bydate" version datasets. Many topics are discussed from gathering data to organizing data with diffent preprocessing and term weighting approaches to perform test by using the feature selection methods and many classification algorithms. The idea behind of feature selection is that determining of the importance of words that are discriminative for categorization task and removing the non-informative terms. In this regard, CHI-Square, Mutual Information, Galavotti-Sebastiani-Simi Coefficient and Document Frequency metrics are studied for feature selection process. The TF-IDF and probability-based term weighting approaches are used to prepare the texts for classification process. Then to get the best achievement for the classifiers and feature selection methods, the effectiveness of system is evaluated with performance evaluation metrics such as accuracy score, precision, recall and f-measure.