Deep convolutional neural network based unconstrained ear recognition

Biometrics is absolutely a critical point on security where human beings interact with machines since it is the key for human identification based on the uniqueness of biometric traits such as fingerprint, iris and ear. Currently, ear biometry is becoming one of the popular research topics on the biometrics and forensic fields. Especially, the ear recognition in-the-wild domain gains importance due to the usability on security purposes and being assistive to the other frequently used biometrics such as face recognition. In addition to being supportive, it may sometimes be an alternative method relative to being more stable over time and not affected from expressions as much as faces. However, ear recognition under unconstrained conditions is a challenging task by reasons of occlusions, variety of head poses, different illumination and low resolution conditions. With ear datasets which were collected under laboratory-like conditions, the high recognition performances are achieved but it is not guaranteed that these type of systems can be robust to the real-world environments. Besides, deep learning models have easily become aware of controlled conditions and obtain high performance on constrained ear datasets that is proven with recent researches. In this thesis, we have comprehensively analyzed ear recognition problem under both constrained and unconstrained conditions. To address this problem, we have benefited from deep convolutional neural networks to enhance the recognition model. We leveraged the ability of deep learning models to learn features for high performance. For this purpose, we used most popular CNN models, which are AlexNet, VGG-16, GoogLeNet, and SqueezeNet. Then, we proposed a domain adaptation approach to improve the performance on UERC dataset which is one of the recently collected unconstrained ear dataset. To enable the domain adaptation, we have generated a new dataset which is one of the biggest ear datasets to the best of our knowledge, from Multi-PIE face dataset and we named it as Multi-PIE ear dataset. Also, we utilized transfer learning method by using pretrained models and we have adapted them to the ear domain. Moreover, we used the common approaches such as data augmentation and data alignment to obtain better results. To improve the performance further, score level fusion with different CNN models has been employed. Besides, one of the developed new loss functions which are center loss and large margin softmax loss have been applied. The domain adaptation technique improved the classification performance around 4% for almost all models. Then, with data augmentation technique, the accuracy is increased 3%. Unlike domain adaptation and data augmentation, the alignment technique did not work in terms of performance gain. Further, with score level fusion approach, we obtained 3-4% performance improvement and we achieved 67.53% accuracy. After, we have achieved 68.18% classification performance with VGG-16 model using center loss. These results show that domain adaptation and data augmentation lead to a significant performance improvement even for unconstrained ear datasets. Further, we analyzed the effect of image quality such as illumination change, resolution and view angle on performance and we have concluded that image quality has significant effect on accuracies. In our last experiment, we have conducted a dataset identification experiment to addressed whether dataset bias problem exists in ear domain. As a result of this, we obtained 99.71% classification accuracy which indicates that there is a strong bias between ear datasets. And this issue has to be investigated further.