Query-by-sign system for Turkish sign language broadcasts

Sign based query search system is a specialized type of query by example searchsystem. The main objective of this study is to locate visual query sign from large videodataset. This study is a baseline of a query by sign search system for deaf and mutepeople to make their access to audiovisual video easy. Hand gesture recognition andretrieval is an open problem and there is no exact solution. In this thesis, our approachproposes a method uses one of the latest successful method OpenPose and subsequeuncedynamic time warping-based retrieval task. We use Turkish Sign Language Broadcastas a dataset and it is processed offline using OpenPose. Hand signs in the datasetand query signs are represented by feature vectors. Feature vectors are a combinationof positions of finger configurations, unit motion vector and appearance based shapeand texture characteristics are which are used to calculate similarity for query match-ing. Cosine metric is used to measure distance to analyze similarity between searchedquery and all subsequences in the dataset. Different sized segmented windows fromdataset are used to compare retrieval performance. Experimental results indicate thatthe proposed method is promising for further studies in query-by-sign search systems.Moreover DTW is combined with and windowing approach improves the performance.Performance of the system is measured by precision at 10 calculation. Number ofsuccessful retrievals from top 10 result give us the performance of the system.