3D Face Recognition using Hyperspectral Images

Face is one of the most common biometric modalities which is used for identification. In this context, face recognition has gained an important role in biometric applications based on identification systems during the last few decades. Since there is no physical interaction required during recognition or identification, it's easy to deploy and implement. In face recognition, a face is categorized as known or unknown by comparing a face with all the faces in a database. Due to inherent distinct features, human face analysis is one of the most effective methods of identifying individuals. Nowadays, utilizing hyperspectral images in face recognition is one of the most important research topics in biometrics, since they contain additional significant spectral information compared to 2D images which have only information in spatial dimensions (texture and structure). A hyperspectral image is a data cube containing two spatial and one spectral dimension. Hyperspectral image samples are captured by a hyperspectral camera which operates at multiple narrow bands within the visible spectrum and neighboring near-infrared spectra. Hyperspectral imaging provides new prospects for improving face recognition accuracy since they contain information in both space and spectral axes. Hence significant information for each person regarding the skin based on reflected, absorbed and released electromagnetic energy at different wavelengths can be extracted. Additional spectral information which is not embedded in traditional grey/color facial images provides an opportunity to improve the recognition accuracy. Hyperspectral imaging employs spatial and spectral relationship simultaneously, which improves segmentation and classification in the respective applications. Difficulties encountered in visible light-based face recognition systems, such as the variance in orientation, illumination or expressions can be minimized by employing hyperspectral imaging. Besides these opportunities, hyperspectral images pose some challenges such as low signal to noise ratios, high dimensionality, and data acquisition needs expensive cameras with multiple sampling in visible and nearinfrared spectra. Despite mentioned challenges, hyperspectral images contain more independent and significant information obtained from different sub-bands than 2D images. Hence, hyperspectral images represented in 3D-cubes are by far more capable in classification processes, which is also ideal for spoofing attacks. In this thesis, we propose novel methods for feature extraction for facial hyperspectral image recognition. The main goal of the thesis is to improve the recognition accuracy of hyperspectral face images. In the first method, three different approaches are proposed employing 3D discrete wavelet transform (3D-DWT) to extract features from the subbands generated by discrete wavelet decomposition. Three approaches include 3D-subband energy (3D-SE), 3D-subband overlapping cube (3D-SOC) and 3D-global energy (3D-GE), which extract different feature vector for each approach containing the energy values calculated from different wavelet sub-bands at different levels of decomposition. Feature vectors generated by three different approaches go through a classifier to complete the face recognition task. In the second proposed method, fusion of spectral information into a single 2D image is achieved by band-specific signal to noise ratio (SNR) based weighting. The fusion method assigns weights based on the calculated band-specific SNR values, weighted sum of the bands generate a single 2D face image. Hence, each pixel along spectral axis is fused to a single pixel resulting a 2D output face image for each 3D hyperspectral face cube. In the third method, in order to fuse spectral bands in hyperspectral face cubes, we apply discrete wavelet transform (DWT) to each pixel along the spectral axis consecutively until the spectral vector for each pixel is decimated to a single pixel transforming the 3D input spectral face image cube into a 2D output image. 2D output images obtained by the second and third methods are processed using principal component analysis method and face recognition is performed with the help of a classifier. The experimental results reveal that recognition accuracy of all proposed methods by using standard hyperspectral databases outperform alternative hyperspectral face recognition of the state-of-the-art methods. Keywords: hyperspectral face image, face recognition, discrete wavelet transform, feature extraction, classification, signal to noise ratio.