Object detection with deep learning-based image preprocessing for autonomous vehicles in foggy weather conditions

Keywords: Autonomous Vehicles, Object Detection, Foggy Weather Conditions, Deep Learning, Dehazing, Computer Vision The rapid advancement of autonomous vehicle technology promises a revolutionary transformation in transportation, with the potential to significantly enhance road safety, improve traffic efficiency, and increase mobility. The foundation of this technology lies in the vehicle's ability to accurately perceive its surrounding environment, make informed decisions, and execute precise maneuvers without human intervention. This perceptual capability is enabled by a sophisticated suite of sensors, including LiDAR, RADAR, and cameras, which work in concert to build a comprehensive model of the world. Among these, camera-based vision systems are indispensable due to their ability to capture rich, high-resolution color and texture information, which is crucial for identifying and classifying a wide range of objects such as other vehicles, pedestrians, cyclists, and traffic signs. The cost-effectiveness and data richness of cameras make them a cornerstone of modern autonomous driving systems. However, the performance of these vision systems is highly susceptible to degradation from adverse atmospheric conditions. While ideal, clear-weather scenarios allow for optimal operation, real-world driving frequently involves challenging environments. Among these, foggy weather presents one of the most significant and dangerous obstacles. Fog is composed of suspended water droplets that scatter and absorb light, leading to a severe degradation of image quality. This manifests as a dramatic loss of contrast, a washing-out of colors, the obscuring of fine details, and the blurring of object boundaries. Consequently, the visibility range is drastically reduced, and the salient features that object detection algorithms rely on become faint or disappear entirely. This degradation is not merely a cosmetic issue; it directly compromises the reliability of the perception system, creating a critical safety risk. An autonomous vehicle that cannot confidently detect a pedestrian or a stopped car in fog is unable to operate safely. Therefore, developing robust perception systems that can overcome the challenges posed by foggy conditions is not just an academic pursuit but a fundamental requirement for the widespread and safe deployment of autonomous vehicles. To address the performance decline of object detectors in fog, significant research has been dedicated to image "dehazing" or "defogging"—a preprocessing step aimed at algorithmically removing the effects of fog from an image to restore its underlying clarity. Historically, many approaches were based on physical models of light scattering, such as the atmospheric scattering model. Techniques like the Dark Channel Prior (DCP) method achieved notable success by estimating the fog density (transmission map) and atmospheric light to invert the degradation process. While foundational, these physics-based methods often rely on assumptions that may not hold true in complex or dynamic scenes (e.g., areas with bright white objects or sky regions), leading to artifacts and color distortions. Furthermore, their computational complexity can be a bottleneck for real-time applications. The advent of deep learning has ushered in a new era of data-driven dehazing techniques. Convolutional Neural Networks (CNNs) and Generative Adversarial Networks (GANs) have demonstrated a remarkable ability to learn the complex, non-linear transformation from a foggy image to its clear counterpart. These models can be trained on large datasets of paired foggy and clear images to produce visually stunning results, restoring vibrant colors and sharp details with high fidelity. However, a critical limitation in many existing deep learning-based systems is their "one-size-fits-all" approach. They often employ static, fixed-parameter filters that are applied uniformly to every input image, regardless of the specific characteristics of the scene. This is a significant drawback because fog is rarely uniform; its density can vary dramatically across a single image and between different scenes. A static filter may be too aggressive for light fog, causing artifacts, or insufficient for dense fog, leaving critical information obscured. This limitation highlights a crucial gap in the literature: the need for adaptive, scene-aware preprocessing. An ideal system should be able to dynamically analyze the content of each specific image—assessing the fog density, contrast distribution, and spatial characteristics—and apply a tailored enhancement. This adaptability is the key to achieving robust performance across the wide spectrum of conditions encountered in real-world driving. This thesis addresses this gap by proposing a system where the preprocessing stage is not a fixed filter but an intelligent, adaptive module designed to work in synergy with a state-of-the-art object detector. This thesis proposes an innovative two-stage, modular system designed to maximize object detection performance in foggy conditions. The core philosophy is to decouple the complex problem into two specialized tasks: adaptive image enhancement and high-performance object detection. This modularity allows each component to be optimized for its specific function and provides flexibility for future upgrades. The system pipeline feeds the enhanced, detail-rich output from the first stage directly into the second, creating a solution that is both powerful and efficient. At the heart of our proposed system is weather-adaptive preprocessing module. Unlike traditional methods, this module does not rely on a fixed set of filtering rules. Instead, it employs a lightweight CNN designed as a "parameter prediction network." For every incoming foggy image, this network performs an analysis to estimate the optimal enhancement parameters tailored to that specific scene. It dynamically assesses the fog's impact on contrast and detail, and then intelligently adjusts the processing pipeline. This process involves predicting key image transformation parameters, such as the gamma correction factor to adjust luminance and contrast, and the unsharp masking intensity to enhance edges and fine details. By making the enhancement process content-dependent, the module provides a scene-specific restoration. For instance, in images with dense fog, it can apply stronger contrast enhancement, while for scenes with light, patchy fog, it can use a more subtle correction to avoid over-enhancement artifacts. The result is a consistently improved image that is rich in detail and structurally faithful to the original clear scene, providing an ideal input for the subsequent detection task. The enhanced images generated by the preprocessing module are fed into a state-of-the-art object detection architecture, YOLOv12m. This model was chosen for its exceptional balance of speed and accuracy, making it highly suitable for real-time autonomous driving applications. The YOLOv12m architecture builds upon the strengths of its predecessors with several key advancements. It features a deep and efficient backbone network for powerful feature extraction, capable of identifying complex patterns even in challenging visual data. Furthermore, it incorporates advanced multi-scale feature fusion mechanisms, such as a Path Aggregation Network (PANet), which allow it to effectively detect objects of various sizes, from large trucks to distant pedestrians. By integrating attention mechanisms, the model can also focus on the most salient regions of the image, further improving its localization accuracy. By training this powerful detector on the clean, high-quality images produced by our adaptive defogging module, we create a synergistic system where both components work together to achieve a level of performance that neither could reach alone. The effectiveness of the proposed system was rigorously evaluated through a series of comprehensive experiments. To ensure a thorough assessment, two distinct and challenging datasets are utilized: FOG-TRAINVAL: A dataset containing a large number of synthetic foggy images, used primarily for training the models. Its diverse scenes and controllable fog levels allow the system to learn to generalize across various conditions. RTTS (Real-world Task-driven Testing Set): A dataset composed of real-world images captured in genuine foggy weather. This dataset serves as the ultimate benchmark for evaluating the system's performance in authentic, challenging driving scenarios. The evaluation was multi-faceted, employing both image quality metrics and object detection performance metrics. Image Quality Assessment: The success of the defogging module was quantified using the Peak Signal-to-Noise Ratio (PSNR) and the Structural Similarity Index (SSIM). Our system achieved an impressive average PSNR of 31.13 dB and an average SSIM of 0.9729, indicating that the enhanced images were not only clearer but also maintained high structural fidelity to the original scenes. Object Detection Performance: The primary goal, object detection, was measured using standard metrics: Precision (P), Recall (R), and mean Average Precision (mAP@0.5:0.95). The experimental results demonstrated the profound impact of our adaptive preprocessing module. On the challenging real-world RTTS dataset, the complete system achieved the scores below: mAP@0.5:0.95: 68.8% Precision: 88.4% Recall: 78.2% This performance represents a significant leap over the baseline YOLOv12m model operating without preprocessing. More importantly, it also surpasses the performance of similar hybrid approaches reported in the literature. For example, our system demonstrated a relative improvement of approximately 14.7% in mAP compared to other strong defogging-plus-detection models. This success validates our central hypothesis: an intelligent, adaptive preprocessing stage that is sensitive to the specific scene conditions can dramatically enhance the capability and reliability of object detection systems in fog. Adverse weather conditions, particularly fog, remain a major impediment to the safe and reliable operation of autonomous vehicles. This thesis aimed to make a significant contribution to the literature by presenting a scene-aware, two-stage modular system that effectively mitigates the impact of fog on camera-based perception. By combining a dynamic, weather-adaptive image preprocessing module with a state-of-the-art object detector (YOLOv12m), our approach enhances both the perceptual quality of the input data and the functional accuracy of the final detection task. The experimental results, validated on challenging real-world datasets, confirm that our system not only achieves a substantial improvement over baseline models but also outperforms similar hybrid approaches. This work provides a robust and effective framework for enhancing the perception systems of autonomous vehicles, thereby increasing their operational safety and reliability in the challenging and unpredictable conditions of real-world driving.