Using deep learning algorithms to identify national role conceptions in Turkish foreign policy (2018 –2023)

Recent advancements in computational power coupled with rapid progress in machine learning and deep learning algorithms have made natural language processing (NLP) tasks feasible across various fields of social science. Despite a growing number of studies using machine learning or deep learning algorithms for text classification topics in diverse domains, there is a gap in the literature on foreign policy analysis in using such techniques. Specifically, studies within the role theory framework – where the researchers mostly focus on how policymakers conceive of their nation's role in international arena – is predominantly done using a hand coding scheme for the systematic text classification. To address this gap and contribute to the field in terms of methodology, this study utilizes deep learning algorithm BERTurk for text classification of Turkish diplomatic text within the framework of role theory. In particular, the research question is framed as "To what extent can the deep learning algorithm BERTurk accurately classify foreign policy text in the context of role theory and national role conceptions?". Methodologically, the research is designed in three steps. First, the data related to Turkish foreign policy – relevant speeches of President Recep Tayyip Erdoğan – from 2018 to 2023 was collected. In the second step, the research data was manually coded using conventional content analysis methods. This hand-coded data was then split (%80-%20) to serve as a ground both for training and testing the model. In the third step, the labeled data was used to feed the model and fine-tuning it with respect to foreign policy texts. Finally, test data was used to analyze the performance of fine-tuned model in predicting the label for each unit of text, employing several metrics such as aggregate accuracy, precision, recall, f1 and MCC along with ROC analysis, PR graph and confusion matrix. The findings show a strong performance of %75 overall accuracy for the BERTurk model in classifying Turkish Foreign Policy-related Political texts.