Object detection on terrestrial imagery using unmanned aerial vehicles

In recent years, rapid advances in artificial intelligence technologies have significantly impacted human life and brought about the emergence and development of Unmanned Aerial Vehicles (UAV) systems, especially in the military field. This thesis focuses on object detection from images obtained by UAVs by utilizing deep learning methods, an artificial intelligence application. Within the scope of the study, the integration of two important areas that today's technology has reached (UAV systems and artificial intelligence) is discussed. Although the study is primarily intended for military use, it also provides a decision support mechanism that can be used in civilian applications. In addition, the system will be able to act autonomously in line with predetermined parameters, even in cases where communication with the relevant systems is not possible. In this respect, the thesis is a roadmap for future research. The most remarkable aspect of the thesis is the process of creating the data set used. Data collection and dataset creation is one of the cornerstones of machine and deep learning projects. This process involves collecting, organizing and processing the data needed to train, validate and test the model. A good dataset plays a vital role in increasing the accuracy and generalization ability of the model. The dataset is composed of real operational environment images obtained through a detailed search on open-source platforms such as Youtube, Twitter and Telegram. With this feature, the developed system is easily adaptable to real operational conditions. Data preprocessing is a set of methods applied to bring data into a suitable form for analysis or modeling purposes or to eliminate potential obstacles. The videos collected in a file that can be used in a file are divided into frames with the CapCut program, and the images that can be used in the data set are saved as photo frames in jpg format, and the noise and unnecessary images on the photos are cleaned by removing them with the Fotor application without disrupting the integrity of the data. For deep learning models that work with image classification, each image must be labeled with a class label. Obtaining accurate results from the studies depends on the correct and meticulous labeling of the data. In the dataset labeling step of our study, the bounding box method was applied to all data using the labelImg 1.8.1 program. The Colaboratory system, a Google service, was used for all training and detection processes with the prepared dataset. Google Colab is a popular cloud-based tool for machine learning, deep learning, data analytics and other Python-based projects. Google Colab works as an online Jupyter Notebook and allows users to write code, run it, create visualizations and annotate it. With this service, the duration of the results obtained can be shortened by selecting high-performance hardware for the GPU feature, which is an important tool especially in visual-based training. With the free version, trainings can be made by allowing up to 12 hours of connection, and high-performance hardware such as Tesla T4 GPU can be used. In the training phase of the study, the You Look Only Once (YOLO) model based on Convolutional Neural Networks (CNN), one of the proven deep learning methods in image processing, was used. Within the scope of the thesis study, YOLOv8 version was used and it stands out as the most widely known and applied version. In the research, a dataset divided into four classes (truck, tank, air defense systems, artillery battery) was prepared. This dataset was divided into 80% training data, 10% validation data and 10% test data and analyzed using the YOLOv8 model. In the later stages of the study, the data set was expanded by applying data augmentation methods and the results obtained with this expanded data set were compared with the initial analysis. Data augmentation is a technique used to improve the performance of machine learning and deep learning models. This method involves creating new data samples by manipulating the existing data set or applying various transformations. Data augmentation methods play an important role in reducing the need for data sets for deep learning models and avoiding problems such as overfitting. Data augmentation aims to reduce overfitting by artificially diversifying the data set so that the model generalizes better to new data, to overcome the problem of insufficient data as it can be difficult to obtain enough labeled data in most applications, and to increase the learning capacity and performance of the model with a more diverse and larger data set. The techniques used on the images can be summarized as rotation and mirroring, which involves rotating the images at certain angles or mirroring them on the horizontal/vertical axis; brightness and contrast adjustment, which involves changing the brightness or contrast levels of the images; cutting, which involves cutting or removing certain regions of the images; and noise addition, which involves adding random noise pixels to the images to create more realistic data. The object detection performance of the applied models was evaluated with metrics such as precision, recall, mAP@50 (mean average precision) and mAP@50-95. mAP is a measure summarizing the success of the model in making accurate object detection. Precision is defined as the ratio of correctly detected objects to total detected objects, while mAP is the average value of this accuracy. mAP@50 refers to the case where the model uses a threshold of 50% to detect objects, i.e. the "Intersection over Union" (IoU) threshold of 50% is used to determine whether the model detected an object correctly. mAP@50-95 refers to the calculation of mAP for cases where the IoU threshold ranges from 50% to 95%. This is an assessment of how accurately the model detects with more stringent thresholds (IoU 50%-95%). That is, the accuracy of the model is measured at different levels of IoU thresholds ranging from 50% to 95%. In the initial model training, precision was 70.6%, recall was 62.1% and mAP@50 was 74.7%, giving a satisfactory initial performance. However, the mAP@50-95 value remained relatively low at 57.1%. The second model training with the augmented dataset resulted in significant improvements in all these metrics. Precision 86.5%, recall 85.5% and mAP@50% 96.6%, which are quite high values. The more challenging evaluation metric, mAP@50-95, showed a significant increase compared to the first training with 62.9%. The findings show that increasing the data collected directly from the field is critical for the success rate of the system and for expanding the number of classes it can recognize. The system, which currently works on truck, tank, air defense systems and artillery battery classes, will be able to recognize other object classes with a high success rate if more images are obtained from the field. In this context, it was concluded that the data collection processes should be systematically expanded in order to increase the object detection and classification success of the system.