Face recognition with local Walsh transform

Face recognition is automatically identifying or verifying a person from a still image or a video frame. The challenges in this area largely occur due to illumination, viewpoint, expression, and resolution variances. There is a wide range of information security, surveillance, law enforcement, and entertainment applications which use face recognition despite there being more accurate biometric indicators such as fingerprint or iris. However, these indicators require the cooperation of those being observed whereas face images can be taken without any cooperation. Face recognition systems generally have four main stages; face detection, alignment, feature extraction, and classification. In this dissertation, we focus on the feature extraction stage, which is probably the most challenging part of face recognition systems. Local approaches have been used frequently due to their success in handling variations such as illumination and expression. Local representations are categorized into two types as sparse and dense. In sparse local representations such as Scale Invariant Feature Transform, points of interest are detected to be used for object detection. In dense local representations, such as Local Binary Patterns and Gabor, features are extracted by applying the method to each pixel of an image. In this dissertation, firstly, we propose a dense local representation which localizes the global Walsh Transform. Secondly, we propose methods to use this representation effectively. The first contribution of this dissertation is a novel dense local representation called Local Walsh Transform (LWT). Walsh Transform (WT) kernel is defined as a set of orthogonal functions with values plus and minus ones. The dot product of any two distinct rows and columns of a WT kernel is zero. Transpose, conjugate, and inverse of a WT kernel are equal to itself. For the WT kernel of order N, if we denote rows by i from 0 to N - 1, the ith row has exactly i zero crossings. In the LWT method, we use this special WT matrix to create filters and apply these filters to each pixels of an image separately. Thus, for a N*N WT we create N*N filters and so N*N images, namely LWT components (y). Based on the fact that using phase information makes the recognition method somewhat more invariant to uniform illumination changes and blurring, we propose an approach to create complex images from LWT components. To this end, we utilize the symmetry between the LWT components with respect to the diagonal. By using this symmetrical relationship, we pair the symmetrical LWT components to create LWT complex images. LWT components which are on the upper side of the diagonal are considered as real part of the complex components, while the components which are on the lower side of the diagonal are considered as imaginary parts of complex images which are paired symmetrically with respect to the diagonal. In addition to the LWT, we propose some additional methods to improve the performance of a face recognition system. The LWT is enhanced with some approaches such as Cascaded LWT (CLWT), dividing image into subregions, weighting, and reducing dimension of features by block-based dimension reduction method. In addition, LWT complex images are converted to grayscale images by XOR approach. For face verification problem, SVM is used for classification, while L2ML is used for dimension reduction. Among the proposed LWT-based methods, Patch-based LWT (PLWT) method enlightens the abilities of the LWT most. In PLWT, we take the advantage of both sparse and dense local representations by applying the LWT method to salient points of face images. In feature extraction stage of the PLWT, first, landmarks of images are detected and some of them are selected. Then, the images and their landmarks may be scaled for once or twice. For each selected landmark at different scales of an image, a patch is created around the landmarks. Following this, the LWT method is applied to all patches of the image and LWT complex images are created. Each complex image is divided into non-overlapping subregions and Phase Magnitude Histogram (PMH) of each subregion is calculated. The PMHs of the subregions are concatenated to create the features of complex images and these features are concatenated to create the features of the patches. Dimensions of the features of the patches are reduced by Whitened Principal Component Analysis method. We choose an unsupervised dimension reduction method to make our methods completely unsupervised and be operable on the databases which do not contain class labels of images. For a test image, the same algorithm is applied to the image and dimensions of the patches are reduced by projection matrices learned in the training stage. Identity of the image is determined according to the similarity between the features of the test image and that of the training set images. The proposed PLWT method is further improved by the CLWT approach. In the CLWT, we apply LWT again to both real and imaginary parts of the LWT complex images. In other words, we once more apply the LWT to sharpened images in which shape information is extracted by the first application of the LWT. This approach gives a depth to the method and increases success rate especially on low resolution images in which shapes and edges are blurred. To use this approach in the PLWT method, all needed is to apply the CLWT instead of the LWT. When the proposed method is carried out with the CLWT approach, we call this method Patch-based CLWT (PCLWT). We show the performance of the proposed methods, by applying them to challenging, widely known, and popular benchmark databases. Specifically, for the face identification problem the Face Recognition Technology (FERET) and the Surveillance Cameras Face (SCface) databases are used, for the face verification problem the challenging Labeled Faces in the Wild (LFW) database is used. The FERET is one of the most widely used benchmarks in the evaluation of face recognition methods. This database gives an opportunity to evaluate methods with various facial expressions, lighting conditions, cameras, scale, and aging property. The images of SCface are taken in an uncontrolled indoor environment that mimics the real world conditions of the indoor face recognition problem. It contains very low resolution images which are taken in the dark. The LFW is a database designed to facilitate the benchmarking of unconstrained pair matching and is also intended to illustrate the problem of learning from one example. This database consists of images collected from the web which contain large pose, occlusion, and expression variations. For most of our experiments, we use the "unsupervised" protocol of the LFW since during both training and testing phases we do not use label information of the subjects, and not access to whether any pair of images is of the same subject. For methods which include SVM or L2ML methods, we use "image restricted with no outside data" protocol of the LFW. In the preprocessing step, images of databases are normalized and aligned by the frontalization method, a similarity transform or the High-Fidelity Pose and Expression Normalization (HPEN) with a similarity transform. The landmarks of the face images are detected by using the Supervised Descent Method (SDM) or Dlib. To the best of our knowledge, the PLWT and PCLWT method achieve the best performance on both the FERET and the SCface databases, and the second best unsupervised category result on the LFW database. Although the other proposed methods are also successful, they are below the state-of-the-art results. In conclusion, the LWT is a strong image descriptor which reveals the shape characteristics of images. The CLWT is especially effective on low resolution images where shape characteristics are blurred. Experimental results show that both the PLWT and PCLWT are robust in terms of illumination, expression, occlusion, and low resolution. The high performances of the proposed methods show that these methods are effective on both wild and low resolution images, and are good alternatives to unsupervised hand-crafted methods.