Using co-training to empower active learning

A huge amount of data is currently available and it is crucial to extract knowledge using this huge amount of data. However, only small portion of this data is classified and labeled as knowledge. Machine learning algorithms are often used to extract knowledge from these datasets; a family of machine learning algorithms that exploit both labeled and unlabeled data to enhance the labeling procedure is called Semi-Supervised Learning. In semi-supervised methods we develop a model, based on scarce labeled instances then, try to expand and improve it using abundant unlabeled instances. Active Learning and Co-training are two prominent members of Semi-Supervised Learning algorithms on which there has been an extensive amount of research. Co-training is the process of learning labels for unlabeled instances from multiple-view datasets. Co-training learns two different classifiers for two different feature views, then, unlabeled instances are labeled when two classifiers are asserted on the class label of an instance. Some of these labeled instances are selected and added to the training set to enhance the learning model. This process is repeated until termination criteria are reached or a classification accuracy is achieved. Meanwhile, Active Learning is a procedure that uses limited human annotator (oracle) knowledge to improve the models. In these algorithms, a plain model is first trained using a small labeled dataset, then some informative unlabeled instances are iteratively selected and labeled by an oracle to improve the model. The most important challenge at this stage is to find the instances which will make optimal improvement over the whole model by knowing their labels. The proposed algorithms in this thesis use Co-training techniques to detect the optimal queries for Active Learning. However, two challenges arise here. Since Co-training uses two independent feature views, first is a need to split feature sets into two different feature views. The second challenge is to select instances to query from oracle using Co-training results. Co-training works based on assumption of having two or more independent and sufficient views. This means that each view of the dataset should be sufficient to train a model and they should be independent from each other. These assumptions are customarily inconceivable for typical single view datasets. Therefore, the best practice to achieve these assumptions is to split the single feature view into two sets of feature views that have the closest amount of information as much as possible. The amount of information is calculated using four methods that are primarily used for feature selection. Previously, the Semi-Supervised Learning combining Co-training with Active Learning (SSLCA) algorithm is only applied Information Gain to attain different feature views. In this thesis, Chi-Square, Analysis of Variance (ANOVA) and ReliefF algorithms are applied for this purpose in addition to Information Gain. For selecting instances from unlabeled dataset to query labels, two approaches are used. First is to calculate the contribution of the instances based on each instance and its neighbors' uncertainty. Second is to use unsupervised learning and clustering uncertain instances to find the best candidate instances for querying. These two approaches form our proposed methods, namely Co-training Enhanced Active Learning (CEAL) and Co-Active Learning. Co-Active Learning was also recently proposed and applied to two feature views. We have extended that algorithm by applying feature splitting so that it can be used in single feature view datasets. Experimental results are conducted over eight benchmark datasets provided by (University of California, Irvine) UCI Machine Learning Repository. These datasets are very common datasets at machine learning research which have been referred and used by hundreds of research. For training models five machine learning algorithms from different types of algorithms have been used. From statistical methods Gaussian Naive Bayes is used that is a generative algorithm. From tree based algorithms Decision Tree and Random Forest have been used. From Neural Networks, Multilayer Perceptrons and from neighborhood based methods K-Nearest Neighbor have been employed. Gaussian Naive Bayes gave us direct statistical certainties and is an iterative fast algorithm. Multilayer Perceptron also provide certainties at a layer before the last layer (softmax). However, other algorithms have been implemented in regression form and by calibrating regression results to be in the range of 0-1 certainty has been inferred. Several tests are performed to compare partitioning methods, query selection strategies and machine learning methods. Classification accuracies are compared with our implementation of SSLCA, Active Learning and Co-training as baseline algorithms. Experimental results show that in most cases CEAL surpass other methods and the best partitioning method was Information Gain with a very slight margin.