Classification of hyperspectral and LIDAR data with deep learning

Remote sensing is a technology which has many military and civil applications. In applications for security, sensitive and reliable imaging systems are needed. Thanks to high end sensors, data obtained by these sensors contains detailed information about the attributes of the observed region. If this rich data which has a complex nonlinear structure is combined with advanced machine learning algorithms, far more information can be inferred compared to a human observer. This thesis aims to classify surface materials over the region of interest with high accuracy by fusing hyperspectral image (HSI) and LiDAR data which are acquired by different types of sensors. To achieve this goal, Deep Learning, which is the most current topic of machine learning, is utilized. In recent years, Deep Learning, where deep network architectures that are constructed with multiple artificial neural layers, is the first option for the solution of classification and detection problems in remote sensing applications. Deep Learning has many subtopics and the Convolutional Neural Network is the main solution of the problem in this thesis. In the experiments, neural network models with different number of layers are tested, and their parameters are optimized according to the test results. Hyperspectral image and LiDAR data are classified with optimized models, and the results are fused by the method proposed in this thesis. For the evaluation of the performance of methods in this thesis, a dataset acquired over urban area in Houston – USA is used. Various preprocessing steps dealing with difficulties of dataset and postprocessing steps to fuse different type of data are needed. First of preprocessing steps is detection and correction of the effect of cloud shadow which degrades performance. Classification accuracy on unprocessed HSI is %77.71 and it increases to %81.49 after shadow correction. Also, the shadow map obtained in this step is used for fusion. In next step where the noise is filtered, Anisotropic Diffusion Filter (ADF) is preferred. Images include sharp edges since data was acquired over urban area and by ADF, edges are preserved while noise is filtered. After filtering, classification accuracy increases to %83.56. EMAP (Extended Morphological Attribute Profiles) technique, which is the last preprocessing task, is used to extract spatial features from LiDAR data. While the accuracy of classification of spatial features is %56.87, fusion method proposed as postprocessing in this thesis is used and accuracy is increased to %89.12. Finally, after applying Majority Voting for regularization, overall accuracy is %93.88.