Environment description with convolutional neural networks

In this study, a system which describes the objects present in the environment and their relative positions has been developed. This system can be used for visually impaired people to learn what is happening in their surroundings and for young children to learn objects in their surroundings. The proposed system can also be used for finding out the names of unknown objects. In the study, the regions that contain objects are detected by the graph based segmentation method in the images. The over segmentation problem of the detected regions is solved using the selective search algorithm. Detected regions are processed with deep convolutional neural network and the names and accuracy probabilities of the objects found in these regions are determined. Finally, the obtained object positions and names are transferred to the user by establishing sentences. Experimental results show that a highly successful environment description tool has been developed.