Video etiketleme için denetimsiz aktif öğrenme

When annotating complex multimedia data like videos, a human expert usually annotates them manually. Even tough manual annotation achieves accurate results, it is a labor-intensive and time-consuming process. On the other hand, computational methods can annotate mass video data for indexing and searching with any or almost no help from human experts effortlessly and faster but they are probably more error prone solutions. The tradeoff between the costs in terms of labor, time and accuracy reveals Active Learning as a natural outcome. Active learning is one of the semi-supervised machine learning methods that benefits from the strongest properties of both manual and computational methods. In an active learning cycle, a learner algorithm discovers the underlying patterns in data and queries the human experts interactively for some informative decision points. It is used when labeled instances are insufficient and acquiring new labels is expensive or especially when unlabeled instances are abundant. In this study, we introduce an unsupervised active learning cycle structure in a flow, which includes clustering, stable matching between the created clusters, various unsupervised selection strategies for selecting the most uncertain and the most certain instances and querying the human annotators. We propose two new cluster selection methods, namely Most Disagreement Selection (MDS) and Hybrid Set Selection (HS), which is a hybrid of MDS and Big Cluster First [2] methods. For MDS and HS, we adopt the "Stable Marriage Problem" solution, in which a stable marriage problem is transformed into a cluster matching problem. We work on REPERE [1] video dataset, which is created for the problem of person identification in videos. Our study aims to identify who is speaking and who is on screen by using multi-modal data. We have evaluated the performance of selection strategies over active learning cycles using multimodality on 28 videos from 7 different TV programs. Each video has three different similarity matrices namely face-to-face, speech-to-speech and face-to-speech. We have run four experiments with regard to matrices in this order: face score for face track annotation, face score for speaker track annotation, speaker score for speaker track annotation and speaker score for speaker annotation.