Visual place recognition with dtw based encoded deep features

Visual Place Recognition (VPR) techniques have opened the possibilities for autonomous robots and driverless cars to localize itself in a cheap and accurate way using only visual input. Previously, sensors-based system, which uses GPS and distance sensors were frequently used. However, its disadvantages such as the cost and the vulnerability to the signal inference, in addition to the quality improvement in the visual sensor (Camera) lead to replacing such systems with visual-based systems. This system-based is capable of getting input rich with information that is important for a wide range of applications including VPR. As a result, many visualization techniques were examined and multiple categories of image descriptors were injected into some localization algorithms, for the purpose of making a system that is able to be aware of the surrounding environment just like humans. In this thesis, a new VPR approach is introduced. This approach uses the Dynamic Time Warping (DTW) and features extracted from a Convolutional Neural Network (CNN) architecture that will be encoded by the Fisher Vector (FV). In more detail, the features are extracted from a pre-trained CNN, then, fed into FV to be encoded and finally pushed to the DTW algorithm that will be used to find the best matches between the reference images and the new coming images (test images). In addition, the performance of different CNN architectures was investigated to find the best architecture fit with DTW, and the performance of all layers from all architectures was compared as well. Furthermore, the advantage of replacing the handcrafted features with deep features was also studied. As the main aim of this work is to develop a robust approach that can face real-life challenges, the deep features are encoded with FV, which we believe can lead to getting more robust features. Our approach was evaluated against other classical approaches, SVM in particular, which was outperformed by our approach especially when it is required to process dataset(s) that has some challenges such as the viewpoint and/or appearance.