Robotic arm trajectory control with reinforcement learning

Robots have become widespread in recent years due to the development of technology, and studies on performing given tasks autonomously are increasing daily. Robot arms, which have the potential to achieve given tasks more powerfully and more precisely than humans, are used in a wide range of areas, such as operating rooms and space missions, as well as industrial facilities. Model-based approaches to the robot arm control problem in the literature require creating a mathematical model expressing the physical system and developing control algorithms. However, these mathematical models are not always easy to obtain, there may be situations such as being unable to solve them, and these methods do not show the desired performance. Artificial intelligence, which is one of the last topics of today's technology, has the potential to provide a solution and convenience for this situation. A practical solution in this area is Reinforcement Learning (RL), a sub-branch of machine learning. Reinforcement learning learns the optimum behaviours through the learning process by making discoveries and evaluating the behaviours revealed in this discovery according to specific criteria, without needing any processed data. Thus, it provides a control option without the need for mathematical models. In this respect, RL has been researched and developed in many areas, from industrial robots to unmanned aerial vehicles. This study used RL algorithms for the robot arm trajectory control problem, regardless of the mathematical model. Deep deterministic policy gradient (DDPG), twin-delayed policy gradient (TD3) and soft actor-critic (SAC) algorithms which are model-free, off-policy, actor-critic reinforcement learning algorithms that developed for continuous state and action spaces, were used. These algorithms were compared among themselves by various parameters such as correct positioning and fulfilment in the desired time in performing tasks through training processes and simulation of trained agents. A second environment with obstacles was created by placing fixed obstacles in the environment where the robot arm is located, and training processes and simulations were repeated for this environment. The effects of a disruptive signal, such as hitting an obstacle in the environment, on the training processes and the behaviour of the trained agents were observed. The agents' preparation, training and testing processes with the algorithms were carried out using the Matlab program. The robot arm model was taken from the Matlab library, and the necessary modifications and environment modelling were made using Matlab's Simulink/Simscape interface. As a result, it has been seen that the algorithms can learn to perform the given tasks with a high success rate.