Image auto-annotation based on combination of text and visual clustering

The emergence of multimedia technology and the rapidly expanding multimedia collections on the Internet have attracted significant research efforts in providing tools for effective retrieval and management of multimedia data. Traditional image retrieval systems were based on manual annotations of images. This is not powerful enough for proper image retrieval, because of manual annotations. Researchers are focused on extracting image features and annotating images automatically by considering their content that are color, texture and shape. Generally, user needs are high level features and retrieval systems consider/process the low-level features to accomplish the retrieval task. This difference between human interpretation and extracted/processed information is known as the semantic gap of such systems. In this thesis, our aim is to find a linkage between low-level features and high level features to bridge the semantic gap. In this thesis, we propose a novel strategy at an abstract level by combining textual and visual clustering results to retrieve images using semantic keywords and auto-annotate images based on similarity with existing keywords. Our main hypothesis is that images that fall in to the same text-cluster can be described with common visual features of those images. In order to implement this hypothesis, we set out to estimate the common visual features in the textually clustered images. When an un-annotated image is given we find the best image match in the different textual clusters by processing their low-level features. Experiments have demonstrated that good accuracy of proposal and its high potential of use in annotation of images and for improvement of content based image retrieval.