Solving perishable inventory control problems in a competitive environment using reinforcement learning

The perishable inventory control problem in a competitive environment is the problem where multiple players compete with other players by managing their own inventory systems. The perishable inventory control problem, which requires a complex sequential decision-making problem with replenishment decisions in each period, cannot be efficiently solved with DP-based equilibrium algorithms due to algorithmic and time complexity. Therefore, algorithms are developed using approximation techniques to solve the problem in this competitive environment. To the best of our knowledge, this is the first study that solves the perishable inventory control problem in a competitive environment with Reinforcement Learning (RL) algorithms. In this thesis, the perishable inventory control problem in a competitive environment is analyzed in cases against fixed replenishment and dynamic replenishment, and equilibrium solutions are obtained for both cases with the RL algorithm. DP-based equilibrium solutions are approximated with algorithms implemented using Nash-Q Learning, one of the techniques in the Multi-agent RL (MARL) world. An experimental design with different parameters is prepared for both cases in a competitive environment. As a result of the tests, the Nash Q-Learning algorithm developed for both cases converges to DP-based equilibrium solutions in a considerably shorter time than the runtime of the DP-based algorithm. In addition, it is observed that the agents trained with the Nash Q-Learning algorithm have a higher win-rate compared to other agents in the competitive environment. Learning rate and learning policy can lead to competitive advantage for the agents.