Temporal anomaly localization

Detecting anomalies in surveillance videos is an important research problem in computer vision. In this thesis, we propose two deep network architectures for anomaly detection, Anomaly Detection Network (ADNet) and Anomaly Detection Network by Object Relations (ADOR). ADNet utilizes temporal convolutions to localize anomalies in videos. The model works online by accepting consecutive windows of video clips. Features extracted from video clips in a window are fed to ADNet, which allows to localize anomalies in videos effectively. We propose the AD Loss function to improve abnormal segment detection performance of ADNet. ADOR employs an object detector and spatio-temporal feature extractor to fuse object relations and action information. Fusion is achieved with cross attention layers which use attention memory from cross encoders. Additionally, we propose to use F1@k metric for temporal anomaly detection. Segment based F1@k is a better evaluation metric than frame based AUC in terms of not penalizing minor shifts in temporal segments and punishing short false positive temporal segment predictions. Furthermore, we extend UCF Crime \cite{sultani2018real} dataset by adding two more anomaly classes and providing temporal anomaly annotations for all classes. Finally, we thoroughly evaluate our model on the extended UCF Crime dataset. ADNet and ADOR produce promising results according to the F1@k metric.