Prioritizing check-worthy claims

The massive amount of misinformation spreading on the Internet on a daily basis has enormous negative impacts on societies. In order to combat against misinformation and its negative outcomes, fact-checking websites detect the veracity of claims . However, fact-checking is an extremely time-consuming process and human fact-checkers are not able detect the veracity of all claims spread on the Internet. Therefore, we need systems to help fact-checkers in the combat against misinformation and to raise public awareness of this important problem. Another problem is that available data resources to develop effective systems are limited and the vast majority of them is for English. In this thesis, we introduce TrClaim-19, which is the very first labeled dataset for Turkish check-worthy claims. TrClaim-19 consists of labeled 2287 Turkish tweets with annotator rationales, enabling us to better understand the characteristics of check-worthy claims. The rationales we collected suggest that claims' topics and their possible negative impacts are the main factors affecting their check-worthiness. In this thesis, we also propose a hybrid model which combines BERT model with various features to prioritize claims based on their check-worthiness. Features we use include domain-specific controversial topics, word embeddings, POS tags, and others. In addition, we explore various ways of increasing labeled data size to effectively train the models such as increasing positive samples, active learning, and utilizing labeled data in other languages. In our extensive experiments, we show that our model outperforms all state-of-the-art models in test collections of CLEF Check That! Lab 2018 and 2019. In addition, when positive samples are increased in the training set, our model achieves the best MAP score reported so far for the test collection of Check That! Lab 2020. Furthermore, we show that cross-lingual training is effective for prioritizing Arabic and Turkish claims, but not for English.