Human based real time surveillance video summarization

In this thesis, a human based video summarization system, which receives video frames from surveillance cameras, was designed and implemented. When an important event happens, sometimes surveillance video records should be examined for hours. At the same time, old entries are deleted at regular intervals because storing of all records is not feasible. In this case it is not always possible to reach the needed records. For these reasons, shortening by summarizing and using only the significant parts of these records have become a necessity.In this work, as a pre-processing step Gaussian Mixture Model is used for foreground extraction. Then the foreground was taken into the frame by using Blob Analysis. Optical flow method is used to obtain information on the object motion within frames.Histogram of Gradients method was applied to motion information of the object for feature extraction.Histogram of gradient features extracted for each object by using Support Vector Machine method and offline generated training set are used to decide the object whether human or not. If the object is human, human tracking is performed with the Kalman Filter. During the video motion information of object has been made continuous by adding time information to histograms with gradient information. Template matching method is applied on the continuous gradient information to find beginnings of movement patterns and beginnings of repetition of the movement patterns. Transition points of the movement patterns are also marked as movement transitions. Movement transitions are the important video moments for the summary video.This study was tested on 4 different datasets. Summarization success rate of the system is the success of detecting periodic movements.First, Weizmann Human Action Database, which is available for public use, is used to test. In this dataset, summary is achived for 6 people with 9 different actions. %89,47 summarization success rate was found in these examples. In the second dataset various situation scenarios that may occur in surveillance video records were performed by 3 different people. As a result of testing on this dataset %93,44 success rate was obtained. %89,29 success rate was achived as a result of test with the third dataset, which contains scenarios similar to the second dataset with a different camera angle. %75,76 success rate is observed with multiple human videos.