Mobile robot application with hierarchical start position deep Q-network algorithm

The remarkable progress of deep learning has also significantly affected the reinforcement learning and resulted in deep reinforcement learning (DRL) method, which is a combination of both methods. DRL does not need a data set and has the potential beyond the performance of human experts, resulting in significant developments in the field of artificial intelligence. However, because a DRL agent has to interact with the environment a lot while it is trained, it is difficult to be trained directly in the real environment due to the long training time, high cost, and possible material damage. Therefore, most or all of the training of DRL agents for real-world applications is conducted in virtual environments. In this study, a DRL agent was trained in a discrete virtual environment with sparse rewards and focused on the real-world targeting problem of a mobile robot using these DRL network parameters. The Minimalistic Gridworld virtual environment was used for the training of the DRL agent, and as far as is known, this study is the first real-world application for the Minimalistic Gridworld virtual environment. A DRL algorithm with higher performance than the classical Deep Q-network algorithm was created with the expanded environment. A low-cost mobile robot was designed for use in a real-world application. In the proposed design, the mobile robot can be controlled from a central computer in a long range. To match the virtual environment with the real environment, algorithms that can detect the position of the mobile robot and the target, as well as the rotation of the mobile robot was created. The model trained in the virtual environment was enabled to be used more efficiently in the real environment. As a result, a DRL-based mobile robot was developed which used only the top view of the environment and could reach its target regardless of its initial position and rotation. In the real environment experiments of this study, the DRL-based mobile robot was started with different initial conditions and it was observed that the mobile robot could successfully reach the target in all experiments. Keywords: Deep reinforcement learning, Deep Q-network, Mobile robot, Object detection