Stance detection in Turkish dataset on Russia-Ukraine war

Social media has evolved into a crucial informational resource to understand public opinion on various issues in recent years. Therefore, the importance of automatic information extraction from these data has increased. Stance detection, one of the subtasks of natural language processing, is also a crucial issue for automatic information extraction. Stance detection automatically determines the user's side regarding a particular subject, event, or person. In this study, a Turkish-labelled data set focusing on the stance determination task to determine social media users' attitudes towards the Russia-Ukraine War was created, and various machine learning methods were evaluated on this data set. For this study, 8215 tweets were collected on Twitter and cleaned. The dataset then was tagged with two targets Russia, and Ukraine. Support Vector Machines, Random Forest, k-Nearest Neighbour, XGBoost, Long-Short Term Memory (LSTM), and Gated Recurrent Unit (GRU) models are employed with GloVe and Fastext word embedding. Since the dataset is unbalanced between the targets, undersampling and oversampling methods were also used with these algorithms. With an F1 score of 0.73 for Russia and 0.81 for Ukraine, the results showed the Support Vector Machines algorithm to produce the best outcomes. In addition to these results, LSTM and GRU also produced outcomes that were highly comparable to those of the Support Vector Machines algorithm. The newly created Turkish corpus can be regarded as a valuable resource for this research area and in the future, transformer-based approach can be used with this corpus. Therefore, this study advances the field of stance detection research using Turkish text.