Data engineering methods for effective fine tuning transformers language models

It is possible to extract features from the text and perform a natural language processing task with traditional methods, but the model performance is limited because the sentence structure and word vectors are not learned enough with the limited amount of labeled data. For this reason, in recent years, researchers prefer to fine-tune pre-trained language models more than traditional methods. Fine-tuning a particular task using a pre-trained transformers language model prepared with large amounts of data yields state of the art results in many natural language processing tasks. Because preparing labeled data is a costly process and labeled data is a limited resource, researchers are examining ways to get better results using less data. For this reason, active learning and weak supervision methods were used as data augmentation methods. At the same time, semi-supervised and unsupervised methods have also been studied research topics. The scope of this thesis is to investigate the most effective fine-tuning method for transformers language models using a limited amount of labeled data. Data engineering methods such as cross-language training, weak supervision, back translation, oversampling, active learning have been used for effective fine-tuning. The subject examined in this thesis is detailed on three different natural language processing tasks. These tasks are detecting check-worthy claims, stance detection, and geolocation detection.