Optimizing service rate of a single server queue usingreinforcement learning

This thesis investigates the optimization of queueing control problems using reinforcement learning (RL), more specifically Q-learning. The study focuses on solving a service rate balancing problem on M/M/1/K queueing systems and then extends to M/G/1/K problems. By using a simulation environment, the Q-Learning Algorithm is trained and compared with optimal results derived from Markov Decision Process (MDP) models. The results indicate that Q-Learning can effectively solve the M/M/1/K problems by creating results that are very close to the optimum. The study explores the effectiveness of different exploration strategies, such as $\epsilon$-Greedy,Boltzmann, and Upper Confidence Bound (UCB) exploration, to improve performance and the convergence rate of Q-learning algorithms. Key findings indicate that Boltzmann exploration strategy perform better than the other strategies, providing a god balance between exploration and exploitation. The study also highlights the sensitivity of Q-learning Algorithm to hyperparameter values, emphasizing the need for careful tuning to achieve optimal performance. The extension of Q-learning to M/G/1/K queueing systems demonstrates the algorithm's adaptability, and the limits and sensitivities of this adaptability. The extension also depicts that embedding model knowledge to the Q-Learning algorithm may not create better results. In conclusion, this thesis validates the potential of reinforcement learning, particularly Q-learning, as an effective tool for optimizing queueing control problems, offering a flexible and adaptable alternative to traditional methods. The results encourage additional research and development of reinforcement learning methods to optimize complex systems and solve harder queueing control problems.