Comparison of Feature Based Fingerspelling Recognition Algorithms

ABSTRACT : Sign language is a manual language which uses hand gestures instead of sounds. These gestures are produced by combining hand-shapes, orientation and movement of hands. Sign language is not international and it has been defined with the intension of communicating with deaf people. In sign language, two major types of communication are considered. The first one is based on word sign vocabulary, where common words are defined by body language. The second, which is also known as fingerspelling is a letter based vocabulary which uses the letters in a particular alphabet and involves the use of hands only. The manual alphabets created for fingerspelling are called finger alphabets. There are two main families of manual alphabets. The one-hand and the two-hand families. American Sign Language (ASL) is used for deaf people in America and south of Canada and it belongs to the one-hand family. The work carried out in this thesis includes the analysis of recognition performance of ASL fingerspelling under four main methods. Mainly the prominent feature extraction, Principal Component Analysis (PCA), Discrete Cosine Transform (DCT) based code assignment and Singular Value Decomposition (SVD) are coupled with circularity. In this work while developing the ASL fingerspelling recognition for the 26 letters of the English alphabet a custom database has been used. This database was generated by using four different signers and each person has signed a total of six times for each letter. Hence 156 images for each signer and a total of 624 images for the entire alphabet were acquired. Each image had 640 × 480 pixel resolution. Throughout the simulations the custom hand dataset and three randomly shuffled versions of this original set were obtained. Each one of the four methods mentioned above had been applied to the individual sets and the results were compared referring to accuracy in determining the signed characters. The simulation results show that when DCT and SVD are applied locally (to sub-blocks instead of the global image) they both give very good performances. In the case of 4:2 training vs. testing the overall recognition rate for the SVD applied locally is 100% and for the DCT applied locally this value was 97.11%. When the SVD is applied globally under the same conditions the overall recognition rate was 92.3%. In fact, SVD using all the singular values has a better performance than the DCT using only the most important coefficients. If we are not concerned about complexity SVD would give the highest overall recognition rate whereas if reduction in complexity is a must DCT is the best contender. The third best result was obtained using the prominent features based method. In contrast, the poorest recognition rate was related to the PCA. The performance of PCA is degraded since hand patterns are not correlated and the mean hand image is quite dispersed. As the training to testing ratio is decreased the overall performance for all methods would gradually also go down. Keywords: American Sign Language, fingerspelling, principal component analysis, discrete cosine transform and singular value decomposition. ……………………………………………………………………………………………………………………………………………………………………………………………………………………