Recognition of human interactions using histogram of sequences

In recent years, many techniques have been proposed for recognition of the human interactions in the computer vision literature. To recognize human interactions, spatial and temporal descriptors have been used together. In modeling of the interactions in the spatial domain, remarkable progress has been made with the experience coming from image processing area. In recognition of the human interactions, modeling temporal information has critical importance, too. It is known that the successful methods that have been proposed in the recent years model both spatial and temporal structure of the motion. In this thesis, we propose a novel representation for human interaction recognition, namely histogram of temporal sequences. The proposed representation consists of the following steps: extracting basic feature descriptors, creating common sequential patterns for each class by using sequence mining, selecting discriminative sequential patterns, creating histograms of sequences from videos and classification. In the sequence mining step, by using hierarchical pyramid structures, the different temporal behaviors of the interactions at different temporal scales are included in the final model. The proposed method can also be used with different type of features. In accordance with this capability, we have tested the performance of the new representation with the deep features. Pre-trained deep features are fine-tuned on the dataset and extracted features are used with the proposed method. We have provided the performance results on two publicly available datasets: UT Interactions Dataset and TV Human Interactions Dataset. In the UT Interactions Dataset, we have the best result on set#2 and the same performance with the best method on the set#1. In the TV Human Interactions Dataset, we have obtained the best results in the literature.