Advanced text classification in natural languageprocessing: The role of transformer

This thesis covers the comparative analysis of traditional methods (Naive Bayes and Support Vector Machines) developed for text classification problems in the field of Natural Language Processing (NLP) and Transformer-based models (BERT, RoBERTa, GPT, T5, DistilBERT) that have shown great success in recent years. The main purpose of the study is to measure the classification performances of these methods on different data sets and to examine the obtained results in detail in terms of accuracy, loss rates and general model success. Within the scope of the thesis, AG News, IMDb and SST-2 datasets were used; text data was subjected to appropriate pre-processing processes and integrated into both traditional and Transformer-based models. While methods such as Naive Bayes and SVM evaluate text features based on basic statistical properties, Transformer-based models benefit from the richness of contextual information thanks to the attention mechanism and large-scale pre-trained language models. In the implementation and experiment phase, each model was trained and tested on the datasets. The results showed that Transformer models generally achieved higher accuracy rates compared to traditional methods, but traditional methods still had an advantage in terms of computational costs and training time. In addition, the classification success of different models varies according to the dataset and problem type; for example, the RoBERTa model achieved the highest accuracy rate on the AG News and SST-2 datasets, while the BERT model stood out on the IMDb dataset. The findings obtained in this thesis are of a guiding nature in model selection for future NLP applications and aim to contribute to both academic and industrial applications. The results also highlight the strengths as well as limitations of the Transformer architecture and provide a basis for developing more efficient and effective methods.