Reinforcement learning based route planning for electric vehicles

The rapid increase in urban populations and the rise of e-commerce activities have introduced significant challenges in last-mile delivery processes. In this context, electric vehicles (EVs) have emerged as an environmentally friendly alternative for last-mile delivery operations. Electric vehicles not only reduce air pollution but also substantially contribute to improving energy efficiency and developing sustainable urban transportation systems. Within this framework, Electric Vehicle Routing Problems (EVRP) have started to replace conventional Vehicle Routing Problems (VRP). Unlike traditional vehicle routing, which typically overlooks refueling constraints, electric vehicle routing problems must consider the locations of charging stations and extended charging durations. This thesis addresses two specific variants of electric vehicle routing problems: the Capacitated Electric Vehicle Routing Problem (CEVRP) and the Electric Vehicle Routing Problem with Pickup and Delivery (EVRP-PDP). For the CEVRP, a novel Q-learning algorithm approaches is proposed. Additionally, for the Capacitated Electric Vehicle Routing Problem with Time Windows (CEVRPTW) and Pickup Delivery Electric Vehicle Routing Problem with Time Windows (EVRP-PDP-TW), a novel Deep Q-Network (DQN) algorithm is developed. To the best of our knowledge, this is the first study employing reinforcement learning methods specifically for the EVRP-PDP-TW. Furthermore, two new datasets are introduced based on real geographic locations from Eskişehir Osmangazi University Meşelik Campus. Optimal solutions obtained through mathematical model are presented for these datasets: the ESOGU-CEVRPTW dataset, applicable to capacitated electric vehicle routing problems, and the ESOGU-EVRP-PDP-TW dataset, applicable to pick up delivery electric vehicle routing problems. The performance of the proposed algorithms is benchmarked against the optimal solutions from the generated datasets. Experimental results indicate that the proposed Q-learning and DQN algorithms achieving near-optimal results within remarkably short computational times. Moreover, once trained, these deep reinforcement learning-based algorithms can be directly applied to new problems drawn from the same distribution without requiring additional training, offering significant advantages over traditional optimization methods. These advantages notably enhance the efficiency of logistics operations.