Development of push-recovery control system for humanoid robots using deep reinforcement learning

The aim of this thesis is to design and implement a completely independent push-and-rescue control system for a bipedal humanoid robot that can imitate the actions of a human. In this study, the thrust-recovery problem of bipedal humanoid robots affected by external forces and thrusts is focused. Since humanoid robots are structurally unstable in terms of balance, this problem emerges as an important problem in robots. In robots, push-rescue controllers consist of 3 strategies: ankle, hip and step. These strategies are biomechanical responses that people show in cases of balance disorder. In this thesis, an active balance control is presented in order to keep the humanoid robots in balance while standing or walking and to prevent balance disorders that may be caused by external forces. In this study, both simulation and real-world tests were conducted. The simulation tests of the study were carried out with 3D models in the Webots environment. Real-world tests were conducted on the Robotis-OP2 humanoid robot. The gyroscope, accelerometer and motor data from the sensors on the robot were recorded and external thrust was applied to the robot. The balance of the robot was ensured by using these recorded data and the ankle strategy. For this, the control of the robot is provided with the PD controller as the classical control method and the Model Predictive Control (MPC) method, which is based on prediction. In addition, Deep Q Network (DQN) and Double Deep Q Network (DDQN) methods from Deep Reinforcement Learning (DRL) algorithms have been applied in order to make the robot fully autonomous. In these applications, both front and rear forces were applied to the robot. In the test studies carried out in the simulation environment, it has been observed that the robot survives the pushes in both cases. The DDQN algorithm gave the best results from the four different control methods applied. The results obtained in the real environment tests showed parallelism with the simulation results.