Evalutation of feature selection and encoding methods for superpixel image parsing

This thesis is about image parsing which is one of the important problems in computer vision. The goal of image parsing is segmentation of object and labeling of each object. Recently, a popular way of image segmentation and classi cation is superpixels. Image is segmented into visually logical small regions by using superpixel algorithm and then, superpixels are parsed into di erent classes. Classi cation performance is signi cantly a ected by the properties of superpixel algorithm and parametric settings. SuperParsing is one of the superpixel-based image parsing algorithm and provides a succesful nonparametric solution for image segmentation and classi cation problem without any need for classi er training. SuperParsing labels each superpixel based on feature matching between the superpixel and a subset of the training superpixels. The training subset is determined by global matching between the test image and the training set. For superpixel matching the method makes use of a rich set of superpixel features. Class conditional log-likelihood is computed based on these matched features. The main objective of this thesis is to show improvements in labeling accuracy percentage by using feature encoding and selection methods, including learned features from Convolutional Neural Network (CNN) models. We perform two different encoding methods to selected features of superpixels and show that feature encoding improves parsing accuracy. The applied feature encoding methods are locality-constrained linear encoding (LLC) and kernel codebook encoding (KCB). LLC encoding method gives us 2:6% improvement on per-pixel accuracy for SIFT Flow dataset and 6:8% improvement on per-pixel accuracy for 19-class LabelMe dataset. KCB encoding method gives us 3:6% improvement on per-pixel accuracy for SIFT Flow dataset and 6:2% improvement on per-pixel accuracy for 19-class LabelMe dataset. All these results are overall improvement which are computed over original SuperParsing. Most recent studies about image segmentation and classi cation use CNN to improve their accuracy percentage. Features extracted from pre-trained networks, which are trained on large image databases, can be used in addition to handcrafted features in image segmentation. Last layer of these CNN models give the best features for classi cation. We test learned CNN features together with KCB or LLC encoding methods. We use CNN features both for global matching and superpixel matching. These tests give us 7:3% overall improvement over original SuperParsing on SIFT Flow dataset and 10:3% overall improvement over original SuperParsing on 19-class LabelMe dataset.