Automated classification of Turkish mobile application reviews

App reviews provide insights into user behavior, ideas, complaints, and overall experience. Product Managers need rapid feedback to ensure customer satisfaction and long-term stay. Hence, app review analysis has emerged as a crucial topic to investigate. Nevertheless, the manual classification of such reviews is costly and demands significant effort. To reduce this cost and effort, automatic classification of app reviews is widely studied in many different languages. Studies focused on detecting Bug Reports, Feature Requests, User Experience, and User Ratings. To the best of our knowledge, our study is the first one that covers these labels and employs the automatic app review classification task in the Turkish language. To accomplish such a task, we manually crawled the Google Play Store and Apple App Store and obtained a dataset of size 5004 from different apps and categories. Additionally, according to the feedback from the industry, we introduce a new label, operational, to identify reviews that are not related to the technical functionality but to the operational processes of the application. Regarding classification, we utilize traditional machine learning techniques with various features, word embeddings, and BERT-based deep language models. Our results show that BERT-based models outperform traditional machine learning techniques. We also applied sampling techniques and observed improvements in the minority classes' performance. Furthermore, we explore the effect of generative artificial intelligence models, specifically GPT-3.5, on our task. We augmented data for a minority class and observed performance improvement with less data creation than sampling techniques. Additionally, we applied prompt engineering (Few-Shot and Chain-of-Thought) and fine-tuning with GPT-3.5 for a minority class. We observed high recall values with Chain-of-Thought prompting and the best F1-score via fine-tuning.