Towards autonomous semantic place cognition in mobile robots

This thesis is concerned with the problem of autonomous semantic place cognition in mobile robots. The importance of this problem is due to the growing demand of using robots for service tasks. This requires the robot to construct a semantic place map, retain this model in its memory and use it as necessary. A semantic map is defined by a labeled graph with nodes corresponding to objects and edges encoding their relations. This thesis addresses this problem through introducing a complete framework for building and using semantic place maps. In the proposed framework, differing from previous work, objects are represented as point cloud objects that are detected through tracking the candidate and thus class-independent point cloud objects over the incoming RGB-D stream. The tracking is done through considering features such as position, shape, color, keypoints and neighboring objects. As the robot is tracking the point cloud objects, it attempts to recognize the respective objects through object recognition. These objects are then used to incrementally build the semantic place map. These maps can then be used in the robot's subsequent visits of the associated place. Experimental evaluation indicates the recognition performance of the semantic place map to depend on both object recognition and routes traversed during first-time visits of places.