Master'sOpen Access

Methods of deceiving transformer language models in stance detection to protect user privacy in social media

Is this your thesis?

This record came from a bulk archive import. If it’s yours, link it to your profile.

2023
0 views
0 downloads

Abstract (EN)

The recent advances in natural language processing have led to a better understanding of language complexities and more successful outcomes in text analysis and comprehension models. Social media platforms, which offer large datasets for natural language processing models, have become an integral part of people's daily lives. Individuals express their emotions, thoughts, experiences, and various personal information through the text they share on social media platforms. However, the ability of artificial intelligence models to track and analyze this data has raised significant privacy concerns among users. In this thesis, we investigate what individuals using social media platforms can do to avoid being detected by artificial intelligence models. We address the task of stance detection on various topics using Turkish and English datasets. We propose methods for BERT and BERTurk-based transformer models to deceive the models by rephrasing and introducing intentional spelling errors. We investigate the effectiveness of the 13 different methods based on their impact on the models' performances. Our experiments demonstrate that intentional spelling error methods significantly reduce the performance of BERT and BERTurk-based models for stance detection. The most effective methods for spelling errors in both languages involve using visually similar characters, splitting words by adding spaces and shuffling the order of letters in words. However, paraphrasing methods are found to be unsuccessful in affecting the models' performances. Two different approaches, manual and automatic, were used for applying the methods. The automatic application of the methods aimed to retain the readability and original meanings of the resulting texts. Two evaluators ensured these checks, and some methods were found to result in reduced readability and changes in semantics due to automatic modifications. Hence, caution is advised in applying shuffle, delete hashtag and adding space. Another point is that in methods based on hashtags, hashtag selections are very important and can cause models to perform better. However, removing or not using hashtags has been more effective in most cases. The proposed methods and the results obtained serve as a guiding reference for users who want to protect their information and privacy from artificial intelligence models.

Author

Dilara Doğan

How to Cite

Dilara Doğan (Master Thesis). Methods of deceiving transformer language models in stance detection to protect user privacy in social media, 2023, TOBB Economics and Technology University.

License

Tüm Hakları Saklıdır

This work is shared under the specified license terms.

More theses from TOBB Economics and Technology University