Trajectory planning based on reinforcement learning for unmanned surface vehicle

The advancement of autonomous systems has enabled unmanned surface vehicles (USVs) to conduct safe and efficient operations in marine environments. In this thesis, an autonomous system is developed to enable USVs to navigate in compliance with the International Regulations for Preventing Collisions at Sea (COLREGs) while minimizing the risk of collisions. The goal is to design a navigation system that performs real-time trajectory planning, avoids collisions, and adapts to environmental conditions. In the developed system, a reinforcement learning method based on the Deep Q-Network (DQN) model is proposed. B-spline curves are used for trajectory generation, modeling the maneuverability of the USV, and producing smooth, adaptive paths through control points. Reinforcement learning-based trajectory planning integrates environmental conditions such as wave height, wind speed, and current velocity. The proposed method was tested in various simulation scenarios, and changes in linear and angular velocity values as well as the theta angle were observed. System performance was evaluated based on Q-values obtained under different learning rates and neural network architectures. Experimental results reveal that the proposed reinforcement learning-based trajectory planning system yields successful outcomes. The system provides direct applicability to similar scenarios with a single training process, offering significant flexibility and speed advantages compared to traditional methods. These features will enhance the operational efficiency of tasks carried out by autonomous maritime vehicles.