Development of a region of interest detection and a generative adversarial network based image augmentation approach for improving facial expression recognition performance

Computer-aided analysis of facial expressions, which is one of the most important tools in interpersonal communication, has become one of the most important research areas of the last decade with the help of technology. The automatic classification of facial expressions is used in many areas such as human-computer interaction, clinical psychology, neurology, security systems, smart environments, customer satisfaction and the advertising industry. In today's world, where imaging and recording operations are carried out at the level of individuals with portable devices, there is a need for applications and methods that can recognize facial expressions quickly and robustly. Within scope of this paper, two different studies were carried out to improve the performance of facial expression recognition. Within the first study, a cropping methodology is proposed for detection of the region of interest on the face that represents the facial expression. Images obtained with this methodology are trained on three convolutional neural networks with each having a different architecture. As a result, important findings were obtained on region of interest and network selection for facial expression recognition. Within the second study, a data augmentation approach based on generative adversarial networks is presented in order to augment a facial expression database that contains a limited number of samples. Augmented images were used to train a convolutional neural network. According to the test phase results of the mentioned network, it was found out that, training with augmented samples that accurately reflects the distribution of the data yields a better performance than the rest of the literature.