Term weighting for text classification

Text classification is the process of assigning text documents to predefined categories. In parallel with rapid development of the Internet and technology, the volume of text documents which are transferred to electronic media has increased dramatically. Hence the importance of organization and classification of text documents and quick accessing to text documents have increased. Since effective vector representations can directly affect the classification performances in text classification, assigning appropriate weight values to the features extracted from text contents is one of the important research problems. Therefore, many term weighting schemes have been proposed in the literature aiming to develop solutions to this research problem. In this thesis, general term weighting problems for text classification and proposed solutions with popular term weighting schemes are extensively analysed and various new solutions are proposed for weighting problems. For this aim, firstly, the effects of reducing high term frequency values with various term frequency factors on the performance of existing supervised term weighting schemes are investigated. In addition, an improved version of recently proposed term weighting approach based on inverse gravity moment has proposed for text classification. Proposed approach presents more reasonable representations for reflecting the discrimination power of terms on some extreme scenarios. Finally, two new term weighting schemes, namely TF-MONO and SRTF-MONO, are proposed for text classification. Proposed schemes can effectively use the distribution information of documents in which terms do not occur. The classification performances of proposed schemes are compared with five popular term weighting schemes by using two classifiers on the three benchmark datasets. Experiment results showed that SRTF-MONO has more successful classification results than other schemes.