Recognition of sign language characters using complex-valued neural networks

Technological advancements play a significant role in the integration of deaf and mute individuals into society. Therefore, further improvements in sign language recognition systems are of great importance. Many studies on sign languages have been conducted using real numbers. In this paper, a new approach is presented for performing feature extraction from images and sign language alphabet recognition using complex numbers. In this context, a model is developed for recognizing American sign language. In the developed model, complex Zernike moments are used to obtain the feature vector of character images. A complex valued deep neural network capable of processing the feature vector composed of complex numbers across layers is also developed. The model achieves recognition rates of 89.01% on the Sign Language MNIST dataset and 98.67% for the holdout and 81.22% for the leave-one-subject-out on the Massey University dataset, respectively, without any preprocessing. The proposed model, which is compared separately with many studies using the same datasets, shows the best performance when the two datasets are considered together. An intelligent system incorporating the proposed method has also been developed.