A novel AI-based autonomous control system for the charging process of electric ships

Maritime transportation constitutes the backbone of global trade, providing indispensable economic and strategic connectivity between nations. With more than 80% of global goods transported by sea, the shipping industry serves as both a key economic driver and a fundamental component of the global supply chain. However, the sector's dependence on fossil fuels has led to significant greenhouse gas emissions, making it one of the major contributors to climate change. This reality has accelerated the search for sustainable and low-emission technological alternatives. In line with this transformation, the International Maritime Organization (IMO) has introduced strict decarbonization targets, including a 50% absolute emission reduction and a 70% carbon intensity reduction by 2050 compared to 2008 levels. The growing adoption of electric and hybrid vessels is a direct response to these global objectives, yet it brings with it a set of operational challenges, among which the development of safe, fast, and fully autonomous charging systems at ports is of critical importance. This doctoral research addresses these challenges by proposing and validating a robotic autonomous control system capable of performing the shore-to-ship (STS) charging process without human intervention. The proposed system combines artificial intelligence, computer vision, and embedded control techniques to execute the detection, localization, tracking, and precise alignment of a ship's charging inlet with the shore-based connector. The primary objectives of the system design are to improve operational safety, reduce connection time, increase alignment precision, and ensure adaptability in diverse environmental conditions. A major contribution of this work lies in the creation of an original charging socket image dataset specifically designed for maritime conditions. The images were captured using an RGB camera mounted on the robotic manipulator under a wide variety of environmental settings, including day and night operations, different weather conditions such as sunny or cloudy skies, and varying illumination intensities. This diversity was deliberately introduced to enhance the robustness of the detection models and to ensure their ability to generalize to real-world port environments where lighting, reflections, and partial occlusions are common. The dataset also incorporates background variations to simulate potential distractors, thereby providing a comprehensive resource for both this project and future research in maritime automation. For the object detection stage, three widely used YOLO architectures: YOLOv3, YOLOv5, and YOLOv8—were implemented and evaluated using standard performance metrics such as precision, recall, mean Average Precision (mAP) at different Intersection over Union (IoU) thresholds, and inference time. The comparative analysis demonstrated that YOLOv5 achieved the optimal balance between detection accuracy and computational efficiency, making it the most suitable choice for real-time embedded deployment in this system. Following the model selection, further performance optimization was conducted by integrating advanced loss functions. Focal Loss was applied to address class imbalance and to prioritize difficult samples during training, SIoU Loss was incorporated to enhance bounding box regression accuracy by accounting for vector-based spatial relationships, and Boundary-Aware Loss (BAL) was used to improve edge precision, especially in high-glare maritime scenes where socket boundaries are less distinct. Experimental results demonstrated that these loss-function-based optimizations improved localization precision and robustness against background clutter. In addition to convolutional network-based models, transformer-based detection architectures such as DETR, RT-DETR, and RF-DETR were also examined. While DETR demonstrated strong capabilities in global feature modeling, RT-DETR was selected for further optimization due to its superior balance between accuracy and inference speed. A novel multi-stage adaptive data augmentation strategy was designed for RT-DETR, which included contrast enhancement using CLAHE to improve performance in low-light environments, shadow manipulation to replicate real maritime lighting variations, mosaic augmentation to increase scene diversity, MixUp and CutMix methods to improve class generalization, and test-time augmentation (TTA) to enhance prediction stability. This adaptive strategy significantly improved model generalization, with mAP scores increasing by more than ten percentage points under challenging lighting and background conditions. After object detection, the system incorporated multi-object tracking to maintain continuous identification of the charging socket and reference points throughout the alignment process. Six state-of-the-art multi-object tracking algorithms—DeepSORT, ByteTrack, OC-SORT, SORT, BoT-SORT, and StrongSORT—were tested and compared. The evaluation used widely accepted metrics such as Multiple Object Tracking Accuracy (MOTA), Identity F1 Score (IDF1), and the number of ID switches. The Singapore Maritime Dataset (SMD) was selected as the benchmark due to its realistic maritime traffic scenes, variety of vessel types, and challenging environmental conditions. ByteTrack and BoT-SORT were found to offer the best combination of accuracy and stability, particularly in scenes with frequent partial occlusions. To further enhance alignment accuracy, a sensor fusion approach was implemented by combining visual data from the RGB camera with distance measurements from a laser rangefinder. After socket detection, the real-time position was transmitted to the programmable logic controller (PLC) via the Modbus protocol, enabling the male and female connectors to be aligned until the distance reached approximately 60 cm. Beyond this point, the laser-based subsystem was activated for fine alignment. Closed-loop feedback control ensured smooth and accurate connector engagement even under environmental disturbances such as vessel movement caused by waves. The entire system was deployed on an NVIDIA Jetson Nano platform, chosen for its low power consumption and high computational efficiency in running CUDA-accelerated deep learning models. Additional optimizations, including model quantization and TensorRT acceleration, were applied to meet the strict real-time inference requirement of under 30 milliseconds per frame. A PyQt-based graphical user interface (GUI) was developed to provide real-time visualization of the detection and tracking outputs, alignment progress, and system diagnostics. The interface included options for manual override to allow human intervention in case of anomalies, as well as logging functions to record operational data for performance analysis and system improvement. Extensive prototype trials were conducted in a laboratory environment simulating port conditions. The trials demonstrated that the alignment and secure connection process could be completed in less than thirty seconds, significantly reducing vessel turnaround times. Detection accuracy remained above 95% mAP at IoU 0.5 in both daytime and nighttime conditions, and tracking stability was maintained despite wave-induced vessel motion and the presence of other moving objects. No safety incidents occurred during the fully autonomous operation cycles. In conclusion, the developed robotic autonomous control system provides a practical and deployable solution for integrating AI-powered shore-to-ship charging into the maritime industry. By combining high reliability, rapid operation, and precise alignment, the system meets industrial performance requirements while contributing to the IMO's greenhouse gas reduction targets. The research offers both academic novelty and direct industrial applicability, representing a significant step towards smart and sustainable port infrastructures that support the widespread adoption of electric vessels.