Deep learning based abnormal situation detection and depth estimation on video images

This thesis, it is aimed to detect abnormal situations recorded by security cameras, locate the abnormal situation in the image, and estimate the distance of the abnormal situation with the depth estimation model by using the deep learning methods. In order to detect the abnormal situation as soon as it occurs, a 3D convolutional neural network and in the continuation a fully connected neural network are used. The 3D convolutional neural network model in the abnormal situation detection system was pre-trained with the sports-1m dataset and the weights obtained as a result of the training were used in the feature extraction process for abnormal situation detection. The features obtained from the 3D convolutional neural network model were applied to the fully connected neural network. A dataset was created for the positioning process and the localization model was trained with this dataset. The images obtained from the input video at certain intervals ere applied to the MobileNet V2 model and the upsampling layers were added to this model for localization estimation so that the related anomaly could be located in the image during the abnormal situation. Input images applied to t e localization model are also applied to the depth estimation model. In the system used for depth estimation, the DenseNet-169 model, which was pre-trained with the ImageNet dataset, works as an encoder. Upsampling decoder model was added to the output of the DenseNet-169 model and a depth map of 320x240 size was produced. The depth estimation model is a pretrained model with the NYU-Depth V2 dataset. In the system developed within the scope of the thesis, the positioning model works as soon as the abnormal situation occurs, and the depth estimation model estimates the distance at which the event occurs. In the thesis study, abnormal situation detection, abnormal situation localization, and estimation of the distance of the abnormal situation are performed simultaneously using video images, this feature makes the thesis work a unique study. In the study, the determination of the relevant parameters that affect the system performance, such as the effect of different optimization methods in the anomaly detectio stage, the effect of different filter sizes in the localization stage, were also carried out and the results were discussed in the relevant section. The data obtained as a result of the training and the results obtained on different video images depending on these training parameters are also discussed in the discussion section. The system can be detected as a result of the thesis study, traffic accidents, etc. while driving in autonomous vehicles. It will be able to detect, locate and estimate the abnormal situation, and can also be used in bus stops, squares, intersections, and other areas where security is required.