Face alignment and frontalization methods for image set based face recognition

Nowadays, the utilization of image sets instead of single face image has become a significant topic due to its enhancement of face recognition performance. The face images within a set are not always captured under ideal pose and lighting conditions. Therefore, preprocessing techniques are necessary to mitigate the effects of adverse conditions such as different pose angles. Face frontalization, which frontalize face images with different pose angles, is one of the important preprocessing steps that improves set-based face recognition performance. In this dissertation, a new face frontalization method was developed by utilizing frontal face images from each sets, and its effects on set-based face recognition performance were analyzed. Technically, a three-dimensional structure and a generative adversarial network model were trained for face frontalization by building upon the "Rotate-Render" method and integrating frontal images into the loss functions. This study compared the performances of the original Rotate-Render (RR) model and the proposed Frontal-based Rotate-Render (FRR) model using statistical metrics on the HONDA/UCSD and ESOGU-285 databases. The "Discriminative Affine and Convex Hull" classifiers, which were proposed within the scope of this dissertation, were utilized to model the features obtained from frontalized face images. The experiments of the dissertation revealed that the set-based method objectively and subjectively improved face recognition performance. As a contribution to the literature, frontal-based face frontalization and high-performance "Discriminative Affine and Convex Hull" classifiers derived from the CNN model's features were proposed.