Data transmission rate based optimal trajectory determination for uav base station using reinforcement learning methods

In this study, optimal trajectory planning is made with the base station (BS) connected to the Unmanned Aerial Vehicle (UAV) in order to increase the service provided to the users. The study is analyzed in two parts: In the first part of the study, UAV-BS serves immobile users with different quality of service (QoS) requirements with its 3-dimensional mobility. While planning the trajectory, the coverage area of UAV-BS and the backhaul capacity between UAV-BS and the Ground Base Station (GBS) are limited. Under these constraints, the goal is to find a trajectory for the UAV-BS that maximizes the total data rate available to users during the flight using reinforcement learning. UAV-BS learns to take the right actions to achieve the desired goal using Q-Learning algorithm in our problem. As a result of trial and error processes with different learning parameters, appropriate parameters are determined and the reinforcement learning model is trained with these parameters. Different communication scenarios are compared to analyze the effects of constraints. Trajectory preferences and total transmission rate changes of UAV-BS are examined according to the effects of the mentioned constraints and heterogeneous QoS demands. Three prominent results demonstrate the effects of coverage, backhaul, and heterogeneous QoS. The UAV-BS tends to increase in altitude as the coverage increases. In addition, the backhaul constraint forces the trajectory of the UAV-BS to approach the GBS. Finally, UAV-BS takes into account the different QoS requirements of users as much as possible. UAV-BS maximizes the total transmission rate by determining the most suitable trajectory to meet these constraints. In the second part of the study, UAV-BS provides services to mobile users by moving at a fixed altitude. Users perform their movement by following a certain pattern. Under these conditions, the maximization of the minimum and maximization problems for UAV-BS are handled, and a suitable trajectory for UAV-BS is sought for the related problem. Deep Q-Learning algorithm is used in our problem due to the changing network topology. The simulation results show that in the maximizing of minimum problem, UAV-BS tries to provide a fair service by balancing its distance to the users, and in the maximization problem, UAV-BS tries to maximize the total amount of service by visiting the places where the number of users are high.