Development of deep learning-based approaches for the detection and distance measurement of rails and surrounding objects

The development of intelligent railway systems increasingly depends on visual perception frameworks capable of executing multiple tasks concurrently and in real time. This thesis proposes a modular deep learning system integrating object detection, semantic segmentation, and depth-aware distance estimation to improve safety, situational awareness, and automation in rail environments. By combining 2D semantic understanding with 3D spatial reasoning, the system enables comprehensive and context-aware scene interpretation tailored to railway operations. The RailSem19 dataset was extended via a novel annotation synthesis pipeline to support both segmentation and detection tasks. State-of-the-art models were benchmarked for each subtask, with top-performing configurations selected using metrics such as mean Average Precision (mAP), mean Intersection over Union (mIoU), and pixel-wise accuracy. Two multi-task fusion strategies were developed: a rule-based inference using segmentation outputs, and a dual-stream architecture processing object- and pixel-level semantics concurrently. The dual-stream model showed improved spatial precision, temporal consistency, and robustness in complex scenarios. Depth perception was achieved using a ZED2 stereo camera, enabling real-time 3D localization. Calibrated depth maps aligned with RGB images allowed the system to estimate distances with an average error below 10% across diverse conditions, supporting reliable detection of pedestrians, vehicles, and obstacles. The system delivered reliable accuracy and real-time performance on CUDA-enabled hardware, demonstrating its applicability to tasks such as railway monitoring, hazard detection, and infrastructure inspection. This work introduces a flexible perception framework tailored for railway automation and establishes a foundation for future advancements in multi-modal fusion, domain adaptation, and deployment in safety-critical environments.