Sensor fusion for gps denied environment for localization and mapping

Unmanned air vehicles (UAVs) are getting more popular recently. UAVs are used on diverse area. Although UAVs are used widely at military area, there are many civil usage, as well. For example, movie production industry, hobby industry. These UAVs seperated two part. One of them is fixed wing and other one is vertical take-off and landing (VTOL) vehicles. VTOL vehicle can take off, hover and land vertically. One of the VTOL vehicle is quadcopter which is used in this study. VTOL vehicle works on a small area, its flight time is limited because of short battery life. However VTOL vehicle is very useful for small area flight because of their maneuvering capabilities. On the other hand fixed wing vehicle is useful for long distance flight. However they can not fly on small areas. For GPS-denied, closed area VTOL vehicle has been chosen for this study. Previously, UAVs can be controlled using a base station. Recently UAVs can be controlled autonomously. Computation power of processors increase with technology. Therefore UAVs do not need base station anymore. They can process data onboard. This increases the autonomy of UAVs. Camera becomes another sensor with increasing processing power and image processing algorithms. Camera can detect face, object recognation, localiation and mapping with using image processing. In this study, firstly UAVs are designed and manufactured as quadcopter which has four rotors. The main reason of designing and manufacturing a custom quadcopter is that we want to build an open source. We can implement any algorithm to quadcopter and control any part of it. Also we want to have a powerfull processor on the quadcopter. So that processor can support algorithms which is complicated and need powerfull processor. Also, this manufactured quadcopter will be used for future academic studies. All parts of quadcopter has been chosen to carry the desired hardware and load. Then these parts are combined and quadcopter take the final form. Some mechanical parts are manufactured from composite metarial to provide ligthness to quadcopter. When quadcopter design, modularity is considered important. Quadcopter is designed as modular, most part of quadcopter can change without dismounting other parts. Naza M2 controller is used for controlling the quadcopter. Naza M2 is ready to use, in future, open source controller like Pixhawk will be used. Test setup is created for testing the controllability of quadcopter and finding PID coefficients of controller. In test setup, quadcopter try to follow specific object autonomously. In order detect the object quadcopter use OpenCV image processing library and detect the object, also find the object location in 2D plane. After finding location of object, quadcopter calculate the location error in X and Y axis. Using PID controller quadcopter follow the object. Quadcopter is succesfuly controlled as semi-autonomously with object tracking algorithm. In order to move autonomously, quadcopter needs to know its location. Also quadcopter needs to calculate distance which it take. For unmanned system localization there are different techniques; odometry, Global Navigation Satellite System (GNSS), inertial navigation and visual techniques. For outdoor environments GNSS information obtained from orbiting navigation satellites provide this information freely. For indoor environments (and at some outdoor areas as well) relying on GNSS is not an option due the fact that the GPS data is not reliable or non-existing due the inability to access enough satellite data. Another approach on localization is to use vehicle tire rotation as well as steering wheel angle. This approach is called odometry. This approach is not possible to use for UAV localization. Because the air vehicle does not have wheels. One other approach is the use of an inertial navigation system (INS), which relies on inertial sensory techniques to estimate vehicle position. In this approach Micro Electro Mechanical systems (MEMS) based micro sensors involving accelerometers, gyros, and magnetometer are combined in a package, and a microprocessor fuses the data through a filter to calculate the vehicle position. If GNSS data is available, the estimated of INS can be updated or calibrated. This approach is called as GNSS/INS integration, and it proposes even smaller errors. Although GNSS based techniques are simple, easy to implement and proposes small localization errors, it is not applicable to our indoor environments due to no GNSS reception. Localization of autonomous air vehicle can also be done using visual information obtained through cameras. Some techniques use artificial visual clues placed on the environment. 2D artificial markers are popular for localization due to fast and robust behavior. Placing unique markers to the know locations of the environment help the vehicle to localize itself. The markers should be in acceptable size, to be seen by the camera in a distance, but not too big to disturb the environment. They should be detected in real-time with an computer. They should be robust to lighting condition changes, which is one of the most limiting factors in successful vision systems in real-world conditions. Placing artificial markers or patterns on an environment for localization is not very practical. Localization of autonomous unmanned air vehicle simultaneously, as it moves using vision and tracking only natural features (such as edges) is called as, simultaneous localization and map building (SLAM). In literature, SLAM (simultaneous localisation and mapping) is well defined problem. Generally radar and sensors are used to solve this problem. Recently, cameras are being used as a sensor for SLAM, due to their low cost compared to LIDAR. Camera has became more popular for SLAM algorithm, because of increasing computation power of processor and accessibility of it. ORBSLAM algorithm is used in this study. ORBSLAM algorithm is very fast and robust SLAM algorithm, when compared other algorithms. The aim of project is that control quadcopter onboard that means control without ground control help. So that ORBSLAM algorithm succesfully implement to Odroid XU4 arm processor. Necessary image is provided from camera which can give high frame per second. ORBSLAM algorithm give good results, when it runs on computer. However if it works on Odroid XU4 it's result is not well like computer. There are some reason of that, algorithm can not find enough feature because of low speed processor on Odroid. Algorithm initializing time takes much time. When algorithm works, there is a breaking points which algorithm fall behind the camera moving speed and can not find new feature points. So algorithm break to run. To solve this problem and obtain more robust data from ORBSLAM algorithm, ORBSLAM compansate with IMU datas using kalman filter. Because IMU sensor can read orientation and accelaration. However, measurement of IMU is not so robust, although measurement is fast. While ORB-SLAM algorithm can read orientation and position values of vehicle more robust than IMU, but it is very slow on embedded system. Because, it needs a lot of computation. Orientations and positions data of two sensor fuse using kalman filter seperately. For orientation structure, orientation datas of IMU sensor and ORBSLAM algorithm are obtained and fused using kalman filter. After that obtained fused orientation data feeds the position fusion. For position structure, position values from ORBSLAM algorithm, acceleration values from IMU sensor and fused orientation values from orientation fusion structure are fused for position fusion. After Sensör fusion algorithm is developed, some experiments are done using different platform. Firstly, sensor fusion algorithm is tried on computer which has i7 processor. Sensör fusion algorith works at i7 computer platform. In this experiment, "L" shaped path is follewed. After experiment, simulation result gives us path measurement estimation. Estimation of path measurement compared with real path measurement and obtained how much algorithm can estimate real path values. Computer which has i7 processor can estimeate real path values very well. Because algorithm can work at this computer with 350 Hz. After algorithm works succeesfully on computer which has i7 processor. Then, algorithm is implemented to embedded platform (Odroid). Because computer is too large to put on board system like quadcopter or mobile platform. We get rid of computer's weight and large size. So we use embedded computer such as Odroid XU4. However, performance of ORBSLAM algorithm performance is decreased, when it runs on Odroid platform. Because Algorithm'speed decreases. Algorithm can work with 8 Hz on Odroid platform. In order to increase performance of algorithm, sensör fusion is applied to ORBSLAM algorithm. Performance of ORBSLAM is increased with sensör fusion. After sensor fusion is applied to ORBSLAM, Performance increasing are obtained, However this inccreasing performance is not enough for real world study. In order to increase the performance of sensor fusion algorithm in next stage of this study, Nvidia tegra TK1 is used. Nvidia tegra TK1 is faster than Odroid and gives much stable and robust result with ORBSLAM algorithm. In future study, performance of localization algorithm can be improved using some improvement. IN order to increase performance of algorithm firstly kalman filter can be modelled as nonlinear system. Also more accurate IMU can be used. These improvement needs much more money, but it gives more reliable results. There is trade off here.