Fusion of Hyperspectral and LIDAR datasets with feature and decision based methods and classification with deep convolutional neural networks

With the evolving next generation remote sensing technology, hundreds of different wavelength images can be captured in the electromagnetic spectrum. In addition to this, light sensing technology can be used to determine distances between objects and distant objects. This information obtained from two different sources constitutes an input to the analysis of the semantic segmentation of a region. In this thesis study, it is aimed to realize the semantic segmentation of two different data sets belonging to the region with HSI (Hyperspectral Image) and LiDAR (Light Detection and Ranging) data with high performance. In particular, classification with deep convolutional neural networks (CNN) has been performed in recent years, except for the classical methods used for classifying hyperspectral datasets, which have had impressive results in the semantic segmentation of images. In the study carried out within the scope of the thesis, the problem is addressed in two steps. First, integration of the hyperspectral and LiDAR information was followed by finalization of the classification. In order to provide additional information on spectral and height information, different structural elements and extended morphological attribute profiles (EMAP) have been specially created for HSI and LiDAR data. Within the scope of integrating the classifier decisions recommended as the first integration method, CNN installations were made specifically for the spectral and morphological profile maps and the results of the classifiers were established. Using the knowledge that the test data provided by the classifier results could be included possibility result in which class, the classifier results were integrated and a general classifier was created. With this method, the best results were obtained in classifying the Muufl dataset. In the dimensional integration, which provides impressive results in the Houston dataset, which is the second method, and in particular the competition dataset, the dimensional integration of different height, morphological and spectral information is provided first. Subsequently, attributes of this integrated data were extracted with filters in AlexNet's first convolution layer. A special CNN setup was performed on this dataset, from which the features were extracted, and the results of the classification were compared with other studies. In the Houston dataset, the highest performance was 93.97% for the cloud-based shadows. Among the studies that did not make any correction to the cloud-based shadowing in the Houston dataset, the highest achievement value of 93.97% was obtained with this method. As deep convolutional neural networks have become very popular in recent years, the feasibility of these networks in many new areas is being questioned. It is one of the important results of this thesis that it can give very effective results in the classification of hyperspectral datasets. However, the need to ensure that formal and spatial information to support spectral information is given as input to CNN is one of the most important results that have been drawn from the study. In addition, the use of filters in the first convolution layer of the AlexNet model classification with CNN has been demonstrated in HSI's.