Deep learning based visual and inertial data fusion for uav localization in case of GNSS signal losses

In recent years, Unmanned Aerial Vehicles (UAVs) have begun to be used in many application areas. In applications where UAVs are used, the accuracy of location information directly affects the success of these applications. Global Navigation Satellite Systems (GNSS) are widely used for location information. However, GNSS is interrupted in closed or difficult environments. In order to prevent this situation and provide a healthy positioning, many methods that can work independently of the GNSS signal have been proposed. Visual and Inertial data have been used separately or combined in GNSS independent methods, which are especially successful in indoor positioning. Although classical image and sensor processing methods were first used for positioning, developments in the field of deep learning have also affected this field. In recent years, deep learning-based methods have been proposed and successful for many operations performed with classical methods in positioning. Although deep learning-based methods have begun to surpass classical methods, the development of both types of methods has continued. In this thesis, Deep Attention-Based Visual Inertial Positioning System has been proposed for UAV positioning in cases where GNSS signal losses occur. A Convolutional Neural Network (CNN) with an integrated attention mechanism is proposed for processing visual data and extracting positioning features. Attention-based Hierarchical Long Short-Term Memory (AHLSTM) model is used for processing inertial data and extracting positioning features. Visual and inertial features obtained with attention mechanisms are integrated with deep learning-based fusion structures to achieve high accuracy location estimation. Using the attention mechanism in visual and inertial data provides a significant performance increase by focusing on prominent regions in terms of location determination. In addition, an original location approach based solely on Inertial Measurement Unit (IMU) data is presented within the scope of the thesis. In this method, IMU data is used to model locational change in space instead of directly producing location information. The developed systems are open source and have been tested on datasets created in a simulation environment. The results show that attention mechanisms provide reliable and highly accurate location independent of GNSS when used with visual and inertial data.