Advantage actor-critic deep reinforcement learning approach for paint shop planning and scheduling

Paint shops usually act as bottlenecks in production facilities requiring a painting procedure. To enhance efficiency and optimize the process by minimizing color batch changes that can decrease productivity, it is essential to develop optimization algorithms. Traditionally, these problems have been addressed using a mixed-integer linear programming (MILP) approach. However, mathematical optimization methods face challenges in adapting to dynamic production planning environments and the real-time nature of a production facility. This is due to its memoryless structure and search for exact and optimal solutions by solving the entire every time a schedule is required. To overcome the issues, this study proposed a deep reinforcement learning algorithm to solve and optimize a paint shop scheduling and planning problem that can adapt to dynamic environments. The actor-critic approach was the best method amongst the other policy-based state-of-the-art deep reinforcement learning algorithms. To train a DRL agent, a real-life simulation model of a paint shop in a household appliance factory was built to act as an environment. After the training and inference processes, the outcome was a paint shop production plan that minimizes the inventory cost and bottlenecks while maximizing the productivity of washing machine production and achieving energy efficiency through planned production stops. Besides some advantages of linear programming methods, DRL models performed well on the selected application of paint shop scheduling and planning problems. It is seen that DRL methods are superior in terms of efficiency and computational performance on inference step while obtaining at least sub optimal solution.