Efficient Multimodal Biometric Systems Using Face and Palmprint

Multimodal biometric systems aim to improve the recognition accuracy by minimizing the limitations of unimodal systems. Fusion of two or more biometric modalities provides a robust recognition system against the distortions of individual modalities by combining the strengths of single biometrics. This thesis proposes different fusion approaches using two biometric systems namely face and palmprint biometrics. These fusion strategies are particularly based on feature level fusion and score level fusion. In this thesis, face and palmprint biometrics are employed to obtain a robust recognition system using different feature extraction methods, score normalization and different fusion techniques in three different proposed schemes. In order to extract face and palmprint features, local and global feature extractors are used separately on unimodal systems. Then fusion of the extracted features of these modalities is performed on different sets of face and palmprint databases. Local Binary Patterns (LBP) is used as a local feature extraction method to obtain efficient texture descriptors and then Log Gabor, Principal Component Analysis (PCA) and subspace Linear Discriminant Analysis (LDA) are used as global feature extraction methods. In order to increase the performance of multimodal recognition systems, feature selection is performed using Backtracking Search Algorithm (BSA) to select an optimal subset of face and palmprint features. Hence, computation time and feature dimension are considerably reduced while obtaining the higher level of performance. Then, match score level fusion and feature level fusion are performed to show the effectiveness and accuracy of the proposed methods. In score level fusion, face and palmprint scores are normalized using tanh normalization and matching scores are fused using Sum Rule method. The proposed approaches are evaluated on a developed virtual multimodal database combining FERET face and PolyU palmprint databases. In addition, a large database is composed by combining different face databases such as ORL, Essex and extended Yale-B database to evaluate the performance of the proposed method against the existing state-of-the-art methods. The results demonstrate a significant improvement compared with unimodal identifiers and the proposed approaches significantly outperform other face-palmprint multimodal systems. Furthermore, we propose an anti-spoofing approach which utilizes both texture-based methods and image quality assessments (IQA) in order to distinguish between real and fake biometric traits. In the proposed multi-attack protection method, well-known full-reference objective measurements are used to evaluate image quality including, Peak Signal-to-Noise Ratio (PSNR), Structural Similarity (SSIM), Mean Squared Error (MSE), Normalized Cross-Correlation (NXC), Maximum Difference (MD), Normalized Absolute Error (NAE) and Average Difference (AD). The three types of feature extraction approaches namely Local Binary Patterns (LBP), Difference of Gaussians (DoG) and Histograms of Oriented Gradients (HOG) are employed as texture-based methods to perform spoof detection in order to detect texture patterns such as print failures, and overall image blur to detect attacks. A palmprint spoof database made by printed palmprint photographs using the camera to evaluate the ability of different palmprint spoof detection algorithms was constructed. We present the results of both face and palmprint spoof detection methods using two public-domain face spoof databases (Idiap Research Institute’s PRINT-ATTACK and REPLAY-ATTACK databases) and our own palmprint spoof database. Keywords: multimodal biometrics, face recognition, palmprint recognition, feature level fusion, match score level fusion, Backtracking Search Algorithm, spoofing, face spoofing detection, palmprint spoofing detection, print-attack, replay-attack