Photometric stereo based 3D face recognition

There are many studies about 2D and 3D face recognition in the literature. 2D face recognition techniques are adversely affected by pose, illumination, expression and make up. 3D face recognition techniques are invariant to posing, illumination and make up. These techniques also solve partially other problems. Reconstruction of 3D face image is practically difficult with 3D scanners in real applications. 3D face images can be acquired with photometric stereo method that uses at least three images that are captured under different illumination conditions. This method is a useful and cost-efficient method that produces 3D face image quickly. In this thesis, easy to use and successful 3D face recognition system that uses 3D face images reconstructed from 2D images using photometric stereo technique is proposed. Yale B and extended Yale B face databases that contain 38 people face images that are captured under 64 illumination directions are used in the experimental studies. All 2D face images have been masked before 3D face reconstruction thus external zone of face and regions that can be created computational errors such as hair have been discarded. 10 image groups were empirically organized with combination of three different 2D images with different illumination directions and 3D test face data has been produced using these images. 5 image groups that 3D data can be produced close to reference 3D image were also selected with combination of three different 2D images with different illumination directions by using Genetic algorithm and 3D test face data has also been produced using these 5 groups. Different face recognition algorithms were tested with these 3D face data in face recognition phase. Firstly, square root of sum of square of differences and 2B correlation methods were implemented directly on height map. Next, ICP algorithm was implemented after converting height map to point cloud. Face surface normal was used by calculating mean of angles between normal of test and reference faces for face recognition. Principal curvatures and their derivatives; mean, Gaussian, curvedness and shape index that described of surface features were used in face recognition process. After SIFT descriptors were extracted from shape index map, these descriptors were matched for recognition phase. When the experimental results were examined, difference and correlation between shape index maps methods achieved over %99 recognition rate. In addition, face recognition were performed with matching SIFT descriptors that were extracted from shape index map without any registration operation in case of absence of some part of the face. According to the results obtained, mouth and nose tip part of the face gave better recognition results than the other face parts. Lastly, recognition tests were made by rotating face parts in some angles. When test results were analyzed, it could be seen that matching SIFT descriptors that were extracted from shape index map were affected little by 90 degree rotations.