On social learning enhanced actor critic reinforcement learning

In this study, a novel method developed by integrating Imitation Learning, a model of Social Learning, into the Actor-Critic Reinforcement Learning algorithm is presented. The aim of this study is to compare the learning speed and solution optimality of agents utilizing the proposed method against those using the standard Actor-Critic algorithm within maze environments characterized by obstacle-free, fixed obstacles, and randomly placed obstacles. The proposed approach combines the Actor-Critic algorithm with a model that imitates observed behaviors under certain conditions, without requiring direct experience transfer between agents. In integrating the imitation feature–the cornerstone of the newly structured method–into the Actor-Critic algorithm, two distinct approaches were adopted: imitation based on a fixed probability value and imitation based on the critic's decision. The impact of the imitation-enhanced Actor-Critic learning method on learning speed, compared to the classical Actor-Critic method, is evaluated through two-dimensional simulations on a maze pathfinding problem. In addition to comparing learning speeds of these methods, the study is supported by results demonstrating the stability of the learning processes and the variation of imitation probability over time. Simulation results show that the proposed learning method significantly accelerates learning overall, achieving optimal results in most maze environments, while the probabilistic imitation method does not perform well in reaching the optimum solution, especially in dynamic obstacle maze environments. However, it was observed that the stability of the Imitation-Enhanced Actor-Critic method presented in the study is higher compared to the standard Actor-Critic approach, with one exception. The findings indicate that integrating imitation learning into the Actor-Critic framework increases convergence to optimum solution and stability of the method, particularly in complex environments. The novel approach proposed in this study has potential applicability in autonomous navigation and robotic systems that require efficient learning through observation.