Semantic concept recognition in digital video and sketch data

Nowadays, video data, which is a multimedia tool, plays an important role in our life and being used commonly in the fields of education, communication, health and history. Furthermore, the proliferation of instant sharing has greatly increased the use of video data in daily life. As a result, there is a need for new techniques in the field of computer vision for the management, classification, detection and retrieval of these data. The most important step to make such functions possible is to estimate and understand the semantic concepts of these data. In recent years, video concept detection has been seen as an important research problem by the multimedia industry. Classification is the most accepted method for concept detection and the outputs of the classification system are interpreted as semantic concepts. These concepts can be used for automatic indexing, search and retrieval of video objects. However, the dimensions of the features used are high and the concept detection with existing classifiers is subject to high computational complexity. In this thesis study; video concept detection based on feature selection and data fusion techniques are proposed in order to enhance the feature effectiveness on the Convolutional Neural Network models for video contents. In order to reduce the cost of training, Principal Component Analysis (PCA) technique is applied at the feature level. Support Vector Machines (SVMs) were used to classify fused feature vectors obtained from different deep convolution neural networks. TRECVID 2013 SIN video data set (38 concepts), which is preferred in past studies, is used in the proposed method for video concept detection. Proposed method for video concept detection is also applied to sketch recognition problem in order to measure the effectiveness of the developed classification method. According to the results obtained, these systems which proposed the data fusion method at the feature level, increased the recognition success.