Computer vision based reuse detection in digital artworks

Image reuse refers to the use of visual elements of existing images in order to create new ones. In this thesis, we study the automatic image reuse detection problem in digital artworks, which is a relatively under-studied problem of image retrieval. We introduce two novel image reuse datasets: an artificial dataset that simulates different types of reuse systematically, and an annotated natural dataset that includes a set of digital artworks that are crawled from the web. Based on the natural dataset, we propose a taxonomy which identifies the primary types of reuse and manipulations. Then, for image reuse detection, we evaluate different feature extraction and classification methods that are commonly used for image copy detection, content-based image retrieval, and computer analysis of artworks. The features we use include, color histograms, Histogram of Oriented Gradient (HOG) descriptors, and the Scale Invariant Feature Transform (SIFT) descriptor and its color-based variants. We use the bag-of-visual-words (BoW) approach with the SIFT descriptors. We also present a novel image description algorithm, called the Affine Invariant Salient Patch (AISP) descriptor, which provides a foreground sensitive description of images by fitting concentric ellipses to the most salient region in an image and extracting features from each track. Our results show that the AISP method can be suitable for reuse detection with its compactness and good retrieval accuracy, especially in images with prominent foreground objects. On the other hand, the use of the SIFT descriptors in a BoW model can be more advisable in a more natural setting and for cluttered scenes.