Face pair matching with local zernike moments and metric learning methods

Since ancient times, human face is used to be one of the most widely used biometrics of human's identity. People have been labeling each other, who are interacted with, by facial appearance. We are still storing other people's faces in our minds to recognize them. There are several biometric identities in human body such as finger print, eyes etc. , but most of them is not suitable to recognize other people in daily lives, except face. With the development of computer technologies and increase in power of microprocessors, applications of face recognition are becoming widespread day by day. There are lots of computer scientist who are interested in face recognition phenomenon. Governments and companies are investing huge amount of money on researching and development of face recognition applications in wide areas such as national security, customer loyalty systems. Face pair matching (FPM) is one the most important subtopics in face recognition problem. FPM is a binary classification problem which is deciding whether or not two face images belong to the same person. Two face images are labeled as match if they belong to same person, otherwise labeled as mismatch. Results of FPM provide very useful information to specific applications like access control of restricted areas or grouping the unknown people's faces in an image gallery. Studies about FPM generally consist of three main steps. First step is detecting the faces, aligning and then cropping. By the help of these preprocessing phases on images, faces become ready to next stages. Second step is feature extraction of these cropped face images. Finally, third step is classification process. In this project, main contribution is focused on second and mostly third stages. In this thesis, Local Zernike Moments (LZM) method is used as feature extraction method. Zernike Moments (ZM) is a feature extraction method that computes complex moments coefficients from all around of image. ZM has a solid performance while using in shape based problems like fingerprint or character recognition, but not inadequate for much more texture based problems like face recognition. Therefore, LZM, a novel face representation method is purposed to use. LZM calculates complex coefficients locally, around the neighborhood of each pixel on image, not globally. Previous works show that LZM is as successful as famous rival methods such as LBP or Gabor on face identification problem. Showing that whether or not LZM may be successful in FPM problem similar to face recognition is one of the main goals in thesis. In LZM method, each moment component produces new complex face images, including real and imaginary parts, from input image. Final feature vector is obtained by concentration of each complex image's phase/magnitude histograms. To sum up, length of the feature vector depends on how many moment component is chosen. So, feature vector may have high dimensions in case of setting the moment parameter high. In order to reducing the processing time, dimension reduction of feature vectors is necessary. Principal Component Analysis (PCA) is used for this purpose. By the help of PCA, length of vectors is decreasing without losing any data variety. After reducing dimensions, there is another method called metric learning which is used for increasing the discrimination power of feature vectors. In metric learning algorithms, a transform matrix is calculated in training stage using vector couples dubbed as match/mismatch by solving a minimizing problem of matching and a maximizing problem of mismatching class distances. In order to apply the metric learning algorithm, transform matrix generated in training stage is multiplied to feature vectors. According to tests, using these statistic based metric learning algorithms on feature vector increase the recognition performance by 6-8%. In this thesis, L2-Norm Metric Learning and Large Scale Metric Learning from Equivalence Constraints algorithms are used. Labeled Faces in the Wild (LFW) image data set is used for benchmarking tests in this work. LFW is a database of face photographs designed for studying the problem of unconstrained face recognition. The data set contains more than 13233 images from 5749 person. Face images of LFW are collected from the Yahoo News website, so images have all of the pose, illumination, angle and aging varieties due to be taken from uncontrolled environment. LFW has two benchmark principles which are restricted and unrestricted settings. All tests must run according to same benchmarking rule. 6000 face image pairs are chosen and they are divided into 10 folds in restricted settings of LFW. Each fold has 300 match and 300 mismatch face pairs. Therefore, success rate of workings are calculated by using cross validation method as suggested. k-Nearest Neighborhood classification method is used for classification in thesis. L1, L2 and Cosine distance metrics are used for calculating distances of feature vectors belongs to images. After applying all algorithms to the input image pairs, distances of the vectors decide whether they are match or mismatch images.