Deep learning based semantic segmentation with RGB+HHA data for obstacle avoidance in mobile robots

The ability of autonomous mobile robots to navigate safely indoors is an important research topic. The robot's ability to avoid obstacles is largely dependent on its accurate perception of the environment. Therefore, the type of raw data used by the robot for environmental perception and the meaningful information derived from this data directly influence its obstacle avoidance performance. Deep learning-based semantic segmentation architectures hold significant potential for obstacle avoidance and navigation. However, studies in this area are still not sufficiently comprehensive, and the visual data used, especially due to the lack of geometric information, yields limited results. This thesis aims to improve the performance of semantic segmentation architectures by utilizing not only image data (RGB) but also geometric features such as horizontal disparity, height above ground, and angle (HHA). This approach aims to model the geometric properties of the scene more accurately, thereby improving both the accuracy of semantic segmentation and the quality of the robot's obstacle avoidance path. To utilize the 6-channel RGB+HHA data type, advanced semantic segmentation architectures have been adapted. Additionally, a local path planning approach for ensuring obstacle avoidance in robots is developed by combining global path planning techniques such as the A* algorithm with the Generalized Voronoi Diagram (GVD) to generate the most efficient local path. Tests conducted using RGB, RGB+Depth (RGB+D), and RGB+HHA data types, obtained from the NYU Depth V2, SUN RGB-D, and Corridor datasets, have been evaluated based on semantic segmentation accuracy (ACC), local path plan quality, and the time required to segment a scene. Experimental results show that the RGB+HHA data type improves semantic segmentation accuracy by at least 1% compared to other data types and provides a 3% improvement in the mean of the Intersection over Union measure (MIoU).