Distributed and scalable deep learning analytics real-time video streaming platform running on gpu servers

This doctoral thesis addresses the critical need for real-time video analytics in areas such as urban security, industrial automation, and autonomous systems. To overcome scalability, low latency, and high efficiency challenges of traditional methods, the DiVA (Distributed Video Analytics) platform was developed. Given the impossibility of processing all complex video analysis scenarios within acceptable timeframes, deep learning-based approaches offer significant advancements. Ongoing research in new deep learning models and analytical platforms continues, and DiVA provides an alternative and effective solution in this evolving landscape. This thesis presents the design, implementation, and performance evaluation of DiVA, a distributed, scalable, deep learning-based real-time video analytics platform. DiVA features a holistic architecture that integrally manages all stages from video stream acquisition to deep learning inference, complex event detection, and automatic scaling. Its model-agnostic architecture allows for easy integration and modification of various deep learning models (e.g., YOLOv8s). This flexibility is crucial for adapting to rapid advancements in AI. DiVA's Kubernetes-based scalable deployment capability ensures system stability and performance under increasing video stream loads, offering a cost-effective operational model. Detailed performance analysis of the core YOLOv8s model within DiVA was conducted across different hardware (CPU and GPU) and software configurations. Findings clearly show GPUs provide significantly higher frame rates (FPS) and lower latency compared to CPUs. Inference frameworks like NVIDIA Triton Inference Server and DeepStream SDK were instrumental in optimizing model serving. Furthermore, the impact of TensorRT quantization (FP32, FP16, INT8) and ONNX optimizations on performance was evaluated. FP16 and INT8 quantization techniques yielded substantially higher inference speeds with acceptable accuracy reductions. DiVA's real-time video analytics capabilities were validated through comprehensive latency and throughput analyses on RTSP video streams, demonstrating its operational effectiveness in real-world scenarios. Beyond object detection, DiVA's Complex Event Processing (CEP) capabilities enable higher-level event detection. The CEP module, integrated with the Temporal Buffer Manager, analyzes temporal and spatial relationships between object detections to automatically identify complex events like prolonged presence, restricted zone entry, or suspicious movement. Comparative evaluations against similar literature systems demonstrated DiVA's competitive performance in both FPS and latency. Notably, DiVA's scalability under increasing video stream loads proved superior. This thesis contributes significantly to real-time video analytics by offering a holistic platform, providing proactive and reactive scaling via its Kubernetes-based Auto-Scaler algorithm, enhancing operational intelligence through CEP and Temporal Buffer Manager integration, and establishing a flexible foundation for future research with its model-independent design.