Theses supervised by Prof. Dr. Fazlı Can
17 theses · İhsan Doğramacı Bilkent University
Sosyal medyadaki kullanıcı topluluklarından yararlanılarak yanlış bilgilerin tespiti
Social media platforms have become a primary source of accessing information. However, the spread of misinformation is inevitable due to the ease of creating and sharing malicious content, including fake news. Social media users in such platforms (e.g., Twitter) often find themselves exposed to similar viewpoints and tend to avoid contrasting opinions, particularly when connected within a community. To investigate this problem, we examine the presence of user communities and leverage them as a tool to detect misinformation on social media. In this thesis, we first collect tweets together with user engagements relevant to the recent events between 2020 and 2022. We then construct a human-annotated social media dataset having 5,284 English and 5,064 Turkish tweets with their veracity labels. After the data construction process, we leverage the presence of user communities for misinformation detection on social media. For this purpose, we propose a text similarity-based method that utilizes user-follower interactions within a social network to identify misinformation content. Our method first extracts important textual features of social media posts using contrastive learning. We then measure the similarity for each social media post, based on its relevance to each user in the community. Next, we train a classifier to assess the truthfulness of social media posts using these similarity scores. We evaluate our approach on three social media datasets and compare our method with the state-of-the-art approaches. The experimental results show that contrastive learning and user communities can effectively enhance the detection of misinformation on social media. Our model can identify misinformation content by achieving a consistently high weighted F1 score of over 90% across all datasets, even employing only a small number of users in communities.
GOOWE ML: Veri akışlarında çok-etiketli sınıflandırma için yeni bir üst-öğrenicili çoklu-sınıflandırıcı
As data streams become more prevalent, the necessity for online algorithms that mine this transient and dynamic data becomes clearer. Multi-label data stream classification is a supervised learning problem where each instance in the data stream is classified into one or more pre-defined sets of labels. Many methods have been proposed to tackle this problem, including but not limited to ensemble-based methods. Some of these ensemble-based methods are specifically designed to work with certain multi-label base classifiers; some others employ online bagging schemes to build their ensembles. In this study, we introduce a novel online and dynamically-weighted stacked ensemble for multi-label classification, called GOOWE-ML, that utilizes spatial modeling to assign optimal weights to its component classifiers. Our model can be used with any existing incremental multi-label classification algorithm as its base classifier. We conduct experiments with 4 GOOWE-ML-based multi-label ensembles and 7 baseline models on 7 real-world datasets from diverse areas of interest. Our experiments show that GOOWE-ML ensembles yield consistently better results in terms of predictive performance in almost all of the datasets, with respect to the other prominent ensemble models.
Evrilen veri akışlarında heyet sınıflandırıcıların anında budanması
Ensemble pruning is the process of selecting a subset of component classifiers from an ensemble which performs at least as well as the original ensemble while reducing storage and computational costs. Ensemble pruning in data streams is a largely unexplored area of research. It requires analysis of ensemble components as they are running on the stream and differentiation of useful classifiers from redundant ones. We present two on-the-fly ensemble pruning methods; Class-wise Component Ranking-based Pruner (CCRP) and Cover Coefficient-based Pruner (CCP). CCRP aims that the resulting pruned ensemble contains the best performing classifier for each target class and hence, reduces the effects of class imbalance. On the other hand, CCP aims to select components that make misclassification errors on different instances. The conducted experiments on real-world and synthetic data streams demonstrate that different types of ensembles that integrate pruners consume significantly less memory and perform significantly faster without hurting the predictive performance.
Kayan pencereler ile güdümsüz kavram sürüklemesinin saptanması: İki yöntem
Data stream mining has become an important research area over the past decade due to the increasing amount of data available today. Sources from various domains generate limitless volume of data in temporal order. Such data are referred to as data streams, and generally, they are nonstationary as the characteristics of the data evolve over time. This phenomenon is called concept drift, and it is an issue of great importance in the literature since it makes models outdated and decreases their predictive performance. In the presence of concept drift, adapting the change in data is necessary to have more robust and effective classifiers. Drift detectors are designed to run jointly with the classification models, updating them when a significant change in the data distribution is observed. In this study, we propose two unsupervised concept drift detection methods: D3 and OCDD. In D3, we use a discriminative classifier over a sliding window to monitor the change in the distribution of data. When the old and the new data are separable with the discriminative classifier, a drift is signaled. In OCDD, we use a one-class classifier over a sliding window. We monitor the number of outliers identified in the sliding window. We claim that the number of outliers are the signs of a new concept, and define concept drift detection as the continuous form of anomaly detection. A drift is signaled if the percentage of the outliers are over a pre-determined threshold. We perform a comprehensive evaluation on the latest and the most prevalent concept drift detectors using 13 datasets. The results show that OCDD outperforms the other methods by producing models with significantly better predictive performances on both real-world and synthetic datasets. D3 is on par with the other methods.
Metin madenciliği ile Türkçede çeviri, sosyal iletişim ve edebi yazı analizi
Text mining is an important research area considering the increase in text generation and the need for analysis. Text mining in Turkish is still not a well-invested research area, compared to the other languages. In this thesis, we analyze different types of Turkish text from different points of views, having an overall review on text mining in Turkish at the end. First, we analyze the translation quality of a Turkish novel, My Names is Red novel, to English, French, and Spanish with the features generated for each chapter. With the proposed method, translation loyalties to the original text can be quantified without any parallel comparisons. Then, we analyze the Turkish spoken texts of 98 people in different age groups in terms of gender and age attributes of the speakers. We also analyze the difference between written and spoken texts in Turkish. Results show that it is possible to predict the attributes of the speaker from the spoken text and written and spoken texts are significantly different in terms of stylometric measures. Later on, we make an assessment on cross-lingual transferring performances of multilingual networks from English to Turkish. We see that transferring is possible; however zero-shot cross-lingual transferring still has its way to be competitive with monolingual networks for Turkish. Lastly, we conduct a time-based stylometric analysis of Ahmet Hamdi Tanpınar's works. We see that Ahmet Hamdi Tanpınar shows some differences compared to his contemporaries.
BELS: Veri akışı sınıflandırması için geniş bir topluluk öğrenim sistemi
Data stream classification has become a major research topic due to the increase in temporal data. One of the biggest hurdles of data stream classification is the development of algorithms that deal with evolving data, also known as concept drifts. As data changes over time, static prediction models lose their validity. Adapting to concept drifts provides more robust and better performing models. The Broad Learning System (BLS) is an effective broad neural architecture recently developed for incremental learning. BLS cannot provide instant response since it requires huge data chunks and is unable to handle concept drifts. We propose a Broad Ensemble Learning System (BELS) for stream classification with concept drift. BELS uses a novel updating method that greatly improves best-in-class model accuracy. It employs a dynamic output ensemble layer to address the limitations of BLS. We present its mathematical derivation, provide comprehensive experiments with 11 datasets that demonstrate the adaptability of our model, including a comparison of our model with BLS, and provide parameter and robustness analysis on several drifting streams, showing that it statistically significantly outperforms seven state-of-the-art baselines. We show that our proposed method improves on average 44% compared to BLS, and 29% compared to other competitive baselines.
Yeni bir sinir topluluğu mimarisi ile gelişen metin akışı sınıflandırması
We study on-the-fly classification of evolving text streams in which the relation between the input data target labels changes over time---i.e. ``concept drift''. These variations decrease the model's performance, as predictions become less accurate over-time and they necessitate a more adaptable system. We introduce Adaptive Neural Ensemble Network (AdaNEN), a novel ensemble-based neural approach, capable of handling concept drift in text streams. With our novel architecture, we address some of the problems neural models face when exploited for online adaptive learning environments. The problem of evolving text stream classification is relatively unexplored and most existing studies address concept drift detection and handling in numerical streams. We hypothesize that the lack of public and large-scale experimental data could be one reason. To this end, we propose a method based on an existing approach for generating evolving text streams by inducing various types of concept drifts to real-world text datasets. We provide an extensive evaluation of our proposed approach using 12 state-of-the-art baselines and eight datasets. Our experimental results show that our proposed method, AdaNEN, consistently outperforms the existing approaches in terms of predictive performance with conservative efficiency.
Çok etiketli veri akışları için denetimsiz kavram kayma tespiti
Many real-world applications adopt multi-label data streams as the need for algorithms to deal with rapidly generated data increases. For such streams, changes in data distribution, also known as concept drift, cause the existing classification models to rapidly lose their effectiveness. To assist the classifiers, we propose a novel algorithm called Label Dependency Drift Detector (LD3), an implicit (unsupervised) concept drift detector using label dependencies within the data for multi-label data streams. Our study exploits the dynamic temporal dependencies between labels using a label influence ranking method, which leverages a data fusion algorithm and uses the produced ranking to detect concept drift. LD3 is the first unsupervised concept drift detection algorithm in the multi-label classification problem area. In this study, we perform an extensive evaluation of LD3 by comparing it with 14 prevalent supervised concept drift detection algorithms that we adapt to the problem area using 12 datasets and a baseline classifier. The results show that LD3 provides between 19.8% and 68.6% better predictive performance than comparable detectors on both real-world and synthetic data streams.
Yerel içerik tabanlı konusal metin bölümlendirme
Understanding the topical structure of text documents is important for effective retrieval and browsing, automatic summarization, and tasks related to identifying, clustering and tracking documents about their topics. Despite documents often display structural organization and contain explicit section markers, some lack of such properties thereby revealing the need for topical text segmentation systems. Examples of such documents are speech transcripts and inherently unstructured texts like newspaper columns and blog entries discussing several subjects in a discourse. A novel local-context based approach depending on lexical cohesion is presented for linear text segmentation, which is the task of dividing text into a linear sequence of coherent segments. As the lexical cohesion indicator, the proposed technique exploits relationships among terms induced from semantic space called HAL (Hyperspace Analogue to Language), which is built upon by examining the co-occurrence of terms through passing a fixed-sized window over text. The proposed algorithm (BTS) iteratively discovers topical shifts by examining the most relevant sentence pairs in a block of sentences considered at each iteration. The technique is evaluated on both error-free speech transcripts of news broadcasts and documents formed by concatenating different topical regions of text. A new corpus for Turkish is automatically built where each document is formed by concatenating different news articles. For performance comparison, two state-of-the-art methods, TextTiling and C99, are leveraged and the results show that the proposed approach has comparable performance with these two techniques. The results are also statistically validated by applying the ANOVA and Tukey post-hoc test. Keywords: Text Segmentation, Topic Segmentation, Natural Language Processing, Lexical Cohesion, Semantic Relatedness.
GOOWE: Değişen veri akışları için geometrik açıdan optimum ağırlıklı çevrim içi çoklu sınıflandırıcı
Designing adaptive classifiers for an evolving data stream is a challenging task due to its size and dynamically changing nature. Combining individual classifiers in an online setting, the ensemble approach, is one of the well-known solutions. It is possible that a subset of classifiers in the ensemble outperforms others in a time-varying fashion. However, optimum weight assignment for component classifiers is a problem which is not yet fully addressed in online evolving environments. We propose a novel data stream ensemble classifier, called Geometrically Optimum and Online-Weighted Ensemble (GOOWE), which assigns optimum weights to the component classifiers using a sliding window containing the most recent data instances. We map vote scores of individual classifiers and true class labels into a spatial environment. Based on the Euclidean distance between vote scores and ideal-points, and using the linear least squares (LSQ) solution, we present a novel dynamic and online weighting approach. While LSQ is used for batch mode ensemble classifiers, it is the first time that we adapt and use it for online environments by providing a spatial modeling of online ensembles. In order to show the robustness of the proposed algorithm, we use real-world datasets and synthetic data generators using the MOA libraries. We compare our results with 8 state-of-the-art ensemble classifiers in a comprehensive experimental environment. Our experiments show that GOOWE provides improved reactions to different types of concept drift compared to our baselines. The statistical tests indicate a significant improvement in accuracy, with conservative time and memory requirements.
Haber akışlarında geçmiş, günümüz ve gelecek: haber zincirlerinin keşfi, anasayfaların haber seçimi, habere karşı toplumsal tepkinin tahmini için mikroblog filtrelenmesi
News streams have several research opportunities for the past, present, and future of events. The past hides relations among events and actors; the present reflects needs of news readers; and the future waits to be predicted. The thesis has three studies regarding these time periods: We discover news chains using zigzagged search in the past, select front-page of current news for the public, and filter microblogs for predicting future public reactions to events. In the first part, given an input document, we develop a framework for discovering story chains in a text collection. A story chain is a set of related news articles that reveal how different events are connected. The framework has three complementary parts that i) scan the collection, ii) measure the similarity between chain-member candidates and the chain, and iii) measure similarity among news articles. For scanning, we apply a novel text-mining method that uses a zigzagged search that reinvestigates past documents based on the updated chain. We also utilize social networks of news actors to reveal connections among news articles. We conduct two user studies in terms of four effectiveness measures: relevance, coverage, coherence, and ability to disclose relations. The first user study compares several versions of the framework, by varying parameters, to set a guideline for use. The second compares the framework with 3 baselines. The results show that our method provides statistically significant improvement in effectiveness in 61% of pairwise comparisons, with medium or large effect size; in the remainder, none of the baselines significantly outperforms our method. In the second part, we select news articles for public front pages using raw text, without any meta-attributes such as click counts. Front-page news selection is the task of finding important news articles in news aggregators. A novel algorithm is introduced by jointly considering the importance and diversity of selected news articles and the length of front pages. We estimate the importance of news, based on topic modelling, to provide the required diversity. Then, we select important documents from important topics using a priority-based method that helps in fitting news content into the length of the front page. A user study is conducted to measure effectiveness and diversity. Annotation results show that up to 7 of 10 news articles are important, and up to 9 of them are from different topics. Challenges in selecting public front-page news are addressed with an emphasis on future research. In the third part, we filter microblog texts, specifically tweets, to news events for predicting future public reactions. Microblog environments like Twitter are increasingly becoming more important to leverage people's opinion on news events. We create a new collection, called BilPredict-2017 that includes events including terrorist attacks in Turkey from 2015 to 2017, and also Turkish tweets that are published during these events. We filter tweets by using important keywords, analyze them in terms of several features. Results show that there is a high correlation between time and frequency of tweets. Sentiment and spatial features also reflect the nature of events, thus all of these features can be utilized in predicting the future.
Kümelemeye dayalı harici intihal ve paralel metin tespit yöntemi
Today different editions and translations of the same literary text can be found.Intuitively such translations that are based on the same literary text are expectedto possess significantly similar structure. In the same way, it is possible that atext that is suspected to have plagiarism can possess structural similarities withthe text that is believed to be the source of the plagiarism. Textual plagiarismimplies the usage of an author?s text, his/her work or the idea that is inserted inanother textual work without giving a reference or without taking the permissionof the original text?s author. Today, existing intrinsic and external plagiarism detectionmethods tend to detect plagiarism cases within a given dataset in order torun these algorithms in a reasonable amount of time. Hence a reference documentset is built in order to search for plagiarism cases successfully by these algorithms.In this thesis, a method for detecting and quantifying the external plagiarism andparallel corpora is introduced. For this purpose, we use the structural similaritiesin order to analyze plagiarism detection problem and to quantify the similaritybetween given texts. In this method, suspicious and source texts are partitionedinto corresponding blocks. Each block is represented as a group of documentswhere a document consists of a fixed amount of words. Then, blocks are indexedand clustered by using the cover coefficient clustering algorithm. Cluster formationsfor both texts are then analyzed and their similarities are measured. Theresults over PAN?09 plagiarism dataset and over different versions of the famousliterary text classic Leyla and Mecnun show that the proposed method successfullydetects and quantifies the structurally similar plagiarism cases and succeedsin detecting the parallel corpora.
Türkçe haber portallarında metin sınıflandırma ve topluluk budama
In news portals, text category information is needed for news presentation. However,for many news stories the category information is unavailable, incorrectlyassigned or too generic. This makes the text categorization a necessary toolfor news portals. Automated text categorization (ATC) is a multifaceted difficultprocess that involves decisions regarding tuning of several parameters, termweighting, word stemming, word stopping, and feature selection. It is importantto find a categorization setup that will provide highly accurate results in ATC forTurkish news portals. Two Turkish test collections with different characteristicsare created using Bilkent News Portal. Experiments are conducted with four classificationmethods: C4.5, KNN, Naive Bayes, and SVM (using polynomial andrbf kernels). Results recommend a text categorization template for Turkish newsportals. Regarding recommended text categorization template, ensemble learningmethods are applied to increase effectiveness. Since they require many computationalworkload, ensemble pruning strategies are developed. Data partitioningensembles are constructed and ranked-based ensemble pruning is applied withseveral machine learning categorization algorithms. The aim is to answer the followingquestions: (1) How much data can we prune using data partitioning on thetext categorization domain? (2) Which partitioning and categorization methodsare more suitable for ensemble pruning? (3) How do English and Turkish differin ensemble pruning? (4) Can we increase effectiveness with ensemble pruningin the text categorization? Experiments are conducted on two text collections:Reuters-21578 and BilCat-TRT. 90% of ensemble members can be pruned withalmost no decreasing in accuracy.
Arama sonucu kümeleme ve etiketlemeye yeni bir yaklaşım
Search engines present query results as a long ordered list of web snippets dividedinto several pages. Post-processing of information retrieval results for easier accessto the desired information is an important research problem. A post-processingtechnique is clustering search results by topics and labeling these groups to reflectthe topic of each cluster. In this thesis, we present a novel search result clusteringapproach to split the long list of documents returned by search engines intomeaningfully grouped and labeled clusters. Our method emphasizes clusteringquality by using cover coefficient and sequential k-means clustering algorithms.Cluster labeling is crucial because meaningless or confusing labels may misleadusers to check wrong clusters for the query and lose extra time. Additionally,labels should reflect the contents of documents within the cluster accurately. Tobe able to label clusters effectively, a new cluster labeling method based on termweighting is introduced. We also present a new metric that employs precision andrecall to assess the success of cluster labeling. We adopt a comparative evaluationstrategy to derive the relative performance of the proposed method with respectto the two prominent search result clustering methods: Suffix Tree Clusteringand Lingo. Moreover, we perform the experiments using the publicly availableAmbient and ODP-239 datasets. Experimental results show that the proposedmethod can successfully achieve both clustering and labeling tasks.
Haber internet sayfaları için şablon-bağımsız içerik çıkartma yöntemi
News web pages contain additional elements such as advertisements, hyperlinks, and reader comments. These elements make the extraction of news contents a challenging task. Current news content extraction (NCE) methods are usually template-dependent. They require regular maintenance, since news providers frequently change their web page templates. Therefore, there is a need for NCE methods that extract news contents accurately without depending on web page templates. In this thesis, a template-independent News content EXTraction approach, called N-EXT, is introduced. It first parses a web page into its blocks according to the HTML tags. Then, it examines all blocks to detect the one that contains the major part of the news content. For this purpose, it assigns weights to the blocks by considering both their textual sizes and similarities to the news title. For quantifying the importance of these two weight components, we use the k-fold cross validation approach; and for assessing the impact of different possible similarity measures, we use a one-way Analysis of Variance (ANOVA) with a Scheff\'{e} comparison. The block with the highest weight is considered as the news block. Our approach eliminates the sentences in the news block that are not related to the news content by considering similarities of sentences to the news block. Finally, it also examines other blocks to detect the rest of the news content. The experimental results show the accuracy and robustness of our method by using two test collections whose web pages are obtained from several different news websites.
Çapraz entropi tabanlı kademeli arama sonuç çeşitlendirmesi
Search engines are used to find information on the web. Retrieving relevant documents for ambiguous queries based on query-document similarity does not satisfy the users because such queries have more than one different meaning. In this study, a new method, cascaded cross entropy-based search result diversification (CCED), is proposed to list the web pages corresponding to different meanings of the query in higher rank positions. It combines modified reciprocal rank and cross entropy measures to balance the trade-off between query-document relevancy and diversity among the retrieved documents. We use the Latent Dirichlet Allocation (LDA) algorithm to compute query-document relevancy scores. The number of different meanings of an ambiguous query is estimated by complete-link clustering. We construct the first Turkish test collection for result diversification, BILDIV-2012. The performance of CCED is compared with Maximum Marginal Relevance (MMR) and IA-Select algorithms. In this comparison, the Ambient, TREC Diversity Track, and BILDIV-2012 test collections are used. We also compare performance of these algorithms with those of Bing and Google. The results indicate that CCED is the most successful method in terms of satisfying the users interested in different meanings of the query in higher rank positions of the result list.
Çok sınıflı dengesiz veri akışlarının sınıflandırılması için dinamik topluluk çeşitlendirme ve kargaşa-tabanlı az örnekleme
The classification of imbalanced data streams, which have unequal class distributions, is a key difficulty in machine learning, especially when dealing with multiple classes and concept drift. While binary imbalanced data stream classification tasks have received considerable attention, only a few studies have focused on multi-class imbalanced data streams. Additionally, dealing with the dynamic imbalance ratio is of great importance. This study introduces a novel, robust, and resilient approach to address these challenges by integrating Locality Sensitive Hashing with Random Hyperplane Projections (LSH-RHP) into the Dynamic Ensemble Diversification (DynED) framework. To the best of our knowledge, we present the first application of LSH-RHP for undersampling in the context of imbalanced non-stationary data streams. The proposed method, undersamples majority classes by utilizing LSH-RHP, provides a balanced training set, and improves the ensemble's prediction accuracy. We conduct comprehensive experiments on 23 real-world and ten semi-synthetic datasets and compare LSH-DynED with 15 state-of-the-art methods. The results reveal that LSH-DynED outperforms other approaches in terms of both Kappa and mG-Mean effectiveness measures, demonstrating its capability in dealing with multi-class imbalanced non-stationary data streams. Notably, LSH-DynED performs well in large-scale, high-dimensional datasets with considerable class imbalances and demonstrates adaptation and robustness in real-world circumstances. For the reproducibility of our results, we have made our implementation available on GitHub.