Fusion of Face and Iris Biometrics for Person Identity Verification

This thesis focuses on fusion of multiple biometric systems in different fusion levels especially score level fusion and feature level fusion. Generally, multimodal biometrics based systems aim to improve the recognition accuracy using more than one physical and/or behavioral characteristics of a person. In fact, fusion of multiple biometrics combines the strengths of unimodal biometrics to achieve improved recognition accuracy. This thesis improves the recognition accuracy by proposing different schemes in score level fusion, feature level fusion, decision level fusion and combination of different fusion levels such as score and feature level fusions. Face and iris biometrics are used to obtain a robust recognition system by using several feature extractors, score normalization and fusion techniques in four different proposed schemes. Global and local feature extractors are used to extract face and iris features separately as unimodal system and then the fusion of these modalities is performed on different subsets of face and iris image databases. Subpattern-based PCA (spPCA), modular PCA (mPCA) and Local Binary Patterns (LBP) methods are used as local feature extractors. Beside these local methods, global feature extractors such as Principal Component Analysis (PCA) and subspace Linear Discriminant Analysis (LDA) are also used to compare the performance of global feature extractors on face and iris images separately. On the other hand, Libor Masek’s iris recognition system is employed on iris images in some schemes to extract iris features. In order to enhance the recognition accuracy of unimodal and multimodal systems in some proposed schemes, Particle Swarm Optimization (PSO) is also implemented as feature selection procedure in reducing the dimension of feature vectors and subsequently improving the recognition performance. The performance of different schemes is validated on several datasets using recognition accuracy and Receiver Operator Characteristics (ROC) analysis. These schemes are based on Weighted-Sum Rule, Sum-Rule, Product-Rule along with Tanh and Min-Max normalization in matching score level fusion. Additionally, FaceFeature Vector Fusion (Face-FVF) or Iris-Feature Vector Fusion (Iris-FVF) with and without PSO feature selection method are used in feature level fusion. Moreover, Majority voting is employed in decision level fusion. The datasets to perform the experiments are selected from ORL, FERET, BANCA, CASIA, UBIRIS and CASIA-Iris-Distance databases. In addition, combination of different databases is used to have different conditions in terms of illumination and pose. Keywords: multimodal biometrics, face recognition, iris recognition, feature extraction, information fusion, Particle Swarm Optimization, match score level fusion, feature level fusion, decision level fusion.