Deep learning based recognition of hand gestures and development of augmented reality application

In recent years, human-computer interaction has become widespread with the developing technology. In particular, augmented reality applications have become a widespread field of study in the field of human-computer interaction. In the field of image processing, there have been significant developments in recent years with the use of deep learning methods. Object recognition, object classification, face recognition, gesture recognition and hand gesture recognition applications developed with deep learning methods are now effectively used in devices used in daily life. In this study, augmented reality technologies that play an important role in human-computer interaction were investigated. A hand gesture recognition model has been developed with deep learning and an augmented reality application that can be controlled by hand gestures has been designed. A hybrid model was created for hand gesture recognition and classification by using convolutional neural network and capsule network algorithm. A dataset called HandGesture14 (HG14) was created, consisting of a total of 14000 images containing 14 different hand gestures. In order to measure the success of the proposed model in object recognition, trainings were carried out on the HG14, Fashion-MNIST and CIFAR-10 datasets using the hybrid model proposed for deep learning and VGG16, ResNet50, DenseNet121, MobileNet, InceptionV3 and CapsNet models. The results of the trainings were compared and the accuracy rates were evaluated. The proposed hybrid model achieved accuracy rates of %90 in the HG14 dataset, %93.88 in the Fashion-MNIST dataset and %81.42 in the CIFAR-10 dataset.