Hand finger recognition from RGB-D video images

Nowadays that computer have been an indispensable part of our lives, developing interfaces that enable body language as an input device as well as mouse and keyboard for human-computer interaction is essential for using the computers more efficiently. Finding the location and shape of the human hand has always been studied in the field of virtual reality, too. Glove based techniques had been widely studied in the early years of the field. Then, image processing techniques color segmentation and background substruction were used. Finally, in the last years of the field infrared camera is started to use. In this thesis study, an application that finds locations and shapes of the open fingers in the depth image taken from RGB-D camera which provides color and depth data was developed. Finding the locations and shapes of the fingers in the image makes fingers available for the design of interfaces that enables human computer interaction. First of all, in this study where Kinect used as RGB-D camera in binary hand image was segmented from the depth image after finding the location of hand that belongs to active user in front of the camera using skeleton data which is produced via processing the depth data of RGB-D camera. For getting better results from binary hand image, binary image was preprocessed and gaps between the fingers were made clear with morphological opening operation. If it is considered that the hand is surrounded by a convex hull, finger tips will lie on the edges of convex hull. Contour tracing followed by convex hull extraction was performed and the edge points that provide some specific conditions of convex hull were extracted as finger tips. Finger skeletones were produced with using the finger tips and wrist point. To recognize the open fingers, characteristic attributes of finger skeletons were extracted. Initial hand image that all the fingers are open in is taken and the characteristic attributes of fingers are introduced to system at the beginning. Open fingers are recognized by comparing characteristic attributes of finger skeletons with the ones that are introduced to system and finding the most similar one.