Self-collision aware reaching and pose control in large workspaces using deep reinforcement learning

Reaching, pose control, and inverse kinematics are fundamental robotic manipulator tasks that underpin other tasks and as such, there is a vast body of related literature from various fields. Control, planning, and more recently learning fields are among the main ones. Traditional control algorithms are prone to failure around singularities and joint limits and do not naturally handle self-collisions. Planning methods are not fast enough for reactive behaviours and require additional infrastructure, including controllers, to work. Learning-based methods have emerged to tackle these issues. However, most of them do not handle arbitrary initial and target poses, ignore self-collisions, do not include orientation information in their targets, work in small workspaces and evaluate themselves with coarse success metrics. In this thesis, we introduce a novel hybrid approach that combines Pseudo-inverse control (PinvC) and model-free reinforcement learning (RL), including state space and reward function design, to fill these gaps in the context of reaching, pose control and inverse kinematics. PinvC already calculates joint velocities given desired task-space (e.g. the end-effector pose) velocities and only requires the kinematic structure of the robot. PinvC is mostly reliable away from joint limits, singularities and when individual links are not prone to collisions. The main idea behind our approach is to use RL to handle these situations. Towards this end, we design a novel state space and reward functions. Our reward function aims to minimize position (reaching task) or pose (inverse kinematics and pose control tasks) errors, reduce self-collisions and reduce joint velocities near the target. Furthermore, we develop a curriculum learning methodology to aid learning. Lastly, we introduce a simple modification, which we call "switching" to further improve task performance. We evaluate our approach with four simulated robots for various problem settings and compare it against traditional and learning-based approaches. Our results show that our approach decidedly outperforms the baselines in terms of mean error, success rates at various thresholds and terminal speed for reaching tasks. In addition, we reduced the number of self-collisions across all the scenarios. Our approach achieved better results when orientation was included, but none of the methods performed very well, especially the learning baselines. We note that the learning-based methods in the literature almost always ignore orientation. As a result, we comprehensively discuss the reasons for orientation failure and potential remedies.