Geometry learning application for school-age children using hand gesture recognition

Hand gesture recognition is a popular research topic which is a great way for implementing Human Computer Interaction interfaces for communicating with a computer without a need for having a mouse or a keyboard. Hand Gesture Recognition has been used on several areas such as daily information retrieval, media player controllers, presentation controllers, communication systems for the hearing impaired, mathematics education, language and culture education and geometry education. In this thesis a geometry learning application for school aged children using HGR is proposed. In this application a video stream is captured from a standart webcam and the received hand gestures are matched with previously defined hand gestures and the respective graphical action for the received hand gesture is given as an output. The system has two main stages. Firstly, the acquisition and interpretation of the image is employed. OxfordNet's VGG-16 architecture was selected as the CNN architecture and was modified in order to fit this study. The data set used to train the model was created by taking photos of 6 different people for each hand gesture. In the second step of the proposed system, the recognized hand gestures are mapped with graphical representations and various interactions. These graphical actions consist of drawing the next and previous shape, enlarging / shrinking / rotating the current shape and showing the perimeter, area and volume formulas for the current shape. The system was tested by adults and school aged children and satisfactory results were obtained.