Unconstrained face recognition under mismatched conditions

Surveillance cameras are very commonplace, and thus can be utilized to make the world a better, safer place. There are many applications which use massive surveillance data to extract information. The speed cameras are mounted on the roads to detect violations in the traffic rules, as well as the security cameras, which are ubiquitous in buildings to detect anomalies. In addition, there is a great interest to analyze and estimate the people's activities in social events. Extensive usage of surveillance cameras has made the monitoring task laborious, such that it is almost impossible for a person to monitor videos efficiently and act accordingly. Therefore, automatic surveillance techniques have been proposed. Face recognition, to a great degree, has been addressed in the literature of computer vision. Face recognition is the problem of matching 1:1 face images (one-to-one), face verification, or 1:N face images (one-to-many), face identification. It has use cases in law enforcement, watchlist, security gates and etc. The usual approach in face recognition is to find the matches in the gallery faces with the probe faces by comparing the distance of the face embedding using a similarity measurement. Conventionally, the extracted features that were used for the comparison were hand engineered, however, due to the rapid progress in the deep learning field, e.g. abundance of the large-scale face database and GPUs, nowadays, deep learning based feature learning methods are preferred. Although, face recognition is a challenging task, as a result of variation in pose, illumination, expression, and occlusion, it is considered to be solved under matched conditions. In matched conditions, face images are literally coming from the same source with relatively high resolution, e.g. faces collected from the Internet. In the mismatched conditions, the face images are coming from the different sources, e.g. surveillance scenarios, where we have the high resolution face images for train set, and the low resolution in the test set. In this thesis, we focused on the face recognition under mismatched conditions and leveraged deep learning models to learn and extract deep face representations. Afterwards, deep face representations are used to compare the faces using a similarity measurement. For this purpose, we used correlation distance between learned features and nearest neighbor classifier to report Rank-1 Identification Rate. In order to extract the features, we have employed 50 and 100 layers residual neural network models trained on VGGFace2 and MS-Celeb-1M databases. We extracted face embedding from the last layer of these networks for each faces in the gallery and the probe set. Furthermore, we experimented with the different amount of information included in the face crops in which we extended the detected bounding boxes which resulted in significant performance boost. Additionally, down-sampling the gallery faces before feature extraction increased the Rank-1 identification rate. For evaluating the performance of the deep learning models at learning discriminative features, we examined the proposed method on ICB-RW and SCFace databases. Our models are trained on VGGFace2 database which is composed of 3.31 million faces of 9131 subjects collected from the celebrity images in different poses and ages and MS-Celeb-1M database which has 10 million images of 100,000 subjects, collected from the celebrity images from the web. The experimental results demonstrate the advantage of using large-scale face database to train deep Convolutional Neural Networks in learning robust face embedding. The experimental results show that quality and variation of the training database is more important than quantity of the database to learn general feature representations. In other words, although, MS-Celeb-1M has 10 times more identity and 3 times more images than VGGFace2, features learned by models trained on VGGFace2 database have better generalization on faces with very low resolution which is a challenging problem in the databases coming from surveillance cameras. Our results on ICB-RW database significantly surpassed the results of previous works, and the experiments on SCFace database achieved state-of-the-art results for distance 3, distance 2, and distance 1 subsets of the probe set. The state-of-the-art results on SCFace benchmark are achieved with an improved version of ResNet-100 trained on MS-Celeb-1M and fine-tuned on VGGFae2 database, which are 76.94% +- 1.98, 98.41% +- 0.92, and 100% +- 0.00 for distance 1, 2, 3, respectively. An ensemble of four model achieved 91.78%, 98.00%, 0.997, Rank-1, Rank-5 IR and area under the curve of cumulative match curve respectively.