Determination of optimal queue policy in M/M/1 queue system with two customer classes

In this thesis, we consider an M/M/1 queue with two different types of customers normal (type 1) and priority (type 2). The two types of customers arrive according to different independent Poisson processes, form a single common queue, and have different holding costs per unit time in the queue. In the system with a single server, when a priority customer enters the queue, he/she is either served according to the order of arrival, or he/she is served by skipping the regular customers. The problem addressed in the thesis is modeled as Markov Decision Process (MDP) and solved by using reinforcement learning algorithms, which is one of the Simulation Based Optimization methods. The goal is to determine the queue policy that approximately minimizes the average cost per unit time. Two different numerical examples are designed, the average costs of policies developed with the reinforcement learning algorithm are compared with myopic policies. These myopic policies; priority customers enter the queue according to the order of arrival, priority customers are brought to the front of the queue, bypassing normal customers, and when priority customers come to the queue, the order of service is randomly decided and they are either moved to the front of the queue or not. According to result of the numerical examples, the average costs per unit time of queuing policies obtained with the reinforcement learning algorithm are much lower than ones of myopic policies.