Analysis of hyperspectral data with involutional neural networks

Hyperspectral Imaging (HSI) is a powerful imaging technique that examines the surface of an object across a wide electromagnetic spectrum, providing spectral information at each pixel. This technology finds widespread use in various fields such as industry, agriculture, environment, and medicine. The widespread use of HSI in different domains emphasizes the importance of accurately analyzing the data. However, the analysis and classification of HSI data face numerous challenges due to their large sizes and high computational costs Overcoming these challenges and effectively analyzing HSI data for diverse applications require precise and reliable methods. Deep learning methods such as Involutional Neural Networks (INNs) can play a significant role in the analysis and classification of HSI data. Involutions are an approach that requires fewer parameters and computational costs compared to traditional convolution-based models while effectively capturing spectral-spatial information. Models developed based on involution use self-attention mechanisms to reduce computational costs when analyzing input data, differing from traditional neural network architectures. This approach proves particularly effective in handling large datasets such as HSI. This PhD thesis examines the use of INNs for the analysis of HSI data and highlights the advantages of this technology. Additionally, it evaluates the potential of involution in HSI based on the results of various studies. Public datasets for remote sensing and earth observation studies were used during these evaluations. For studies in the biomedical field, the neonatal HSI data collection created under a TÜBİTAK project was employed. Involution-based models were developed to analyze HSI data in these fields, and comprehensive analyses were conducted using various performance metrics. In the initial study, experimental investigations were conducted using publicly available Indian Pines (IP), Pavia University (PU), Salinas Scene (SA), and Kennedy Space Center (KSC) datasets in the fields of remote sensing and earth observation. This study focuses on the classification of HSI by a unique neural network model integrating the concept of involution, known as the Involutional Residual Spectral Network (IRSN). This model aims to reduce the computational cost of convolution-based methods by adopting the involution approach, which enhances classification accuracy while ensuring computational efficiency. According to the experimental results, the developed IRSN model emerged as highly suitable for HSI datasets and stood out as a superior model in terms of performance. In another study, the involution-based HarmonyNet model was developed to assess the health status of neonates, and a comprehensive evaluation was conducted. Assessments conducted on an HSI dataset comprising 110 patient cases and 110 healthy cases demonstrate that the HarmonyNet model achieved significant success with high accuracy, reliability, and low computational cost. The analyses indicate that optimizing hyperparameter settings within the involution design is crucial for enhancing the model's performance. Additionally, an ablation study confirmed that the model's success stems from the synergistic integration of different features and underscored the model's broad applicability. Another study aims to evaluate the effectiveness of the involution-based MC-I2Net model, which incorporates multiscale contextual structure and inception modules, developed for the detection of neonate diseases. Experiments and analyses demonstrate that the MC-I2Net model exhibited significant performance in classifying 240 HSI data belonging to five diseases (Respiratory Distress Syndrome, Necrotizing Enterocolitis, Intracranial Hemorrhage, Aortic Coarctation, and Pneumothorax) and one control group, achieving higher success metrics compared to other convolution-based models. Although the involution-based MC-I2Net has fewer parameters and FLOP values compared to convolution-based methods, inference time was longer in some cases. However, this study emphasized the importance of selecting specific parameters for optimal performance. Analyses conducted to enhance the model's performance highlighted the significant role of the inception module and multiscale contextual structure. Additionally, it was emphasized that the involution-based MC-I2Net requires less computational power and is a suitable option for practical applications due to its high performance. In another study conducted within the scope of the thesis, a novel hybrid model is proposed for the diagnosis and monitoring of neonate health. Beyond the classification experiments traditionally conducted using only spectral data, a more effective approach was achieved by incorporating blood biomarkers such as hemoglobin and bilirubin. The HybridCISN model, incorporating both 2D/3D convolutional structures and involution structures for the evaluation of spectra and blood biomarkers together, was developed. Two different approaches were adopted in this study. In the single spectrum evaluation approach, only the HybridCISN model was used. In the joint evaluation approach, features were extracted from the spectrum data using the convolution and involution layers within HybridCISN, and blood biomarkers were added to the model after the flattening stage. Different classifiers were used at this stage, and their performances were evaluated. This approach enabled the evaluation of spectra and blood biomarkers together. Binary and multi-classification experiments were conducted. Experimental results indicate a significant improvement in the model's performance with the addition of blood biomarkers to the spectral data. Additionally, ablation analyses underscored the importance of the involution layer in evaluating the model's components and suggested a focus on this layer for enhancing the model's performance in the future. In the fifth study within the thesis, the analysis and classification of neonate HSI data were conducted. This study targeted the evaluation of various dimensionality reduction techniques. It was determined that the examined dimensionality reduction techniques played a significant role in HSI data analysis and yielded more effective results when combined with INNs. The performance of different techniques was examined, and methods such as Non-negative Matrix Factorization (NMF) and Principal Component Analysis (PCA) stood out for their ability to accurately classify samples with different class labels and make correct predictions across all test data. This study contributed to identifying effective and reliable dimensionality reduction methods for HSI data analysis. These all studies highlight the advantages of using INNs in the analysis of HSI data. INNs attract attention for their ability to capture spectral-spatial information effectively while requiring fewer parameters and computational costs compared to traditional methods. Particularly, these studies have pioneered the development of a novel approach for neonates worldwide and introduced a comprehensive data collection created under a TÜBİTAK project for the first time. The studies indicate that HSI technology has become a significant tool in various fields such as industry, agriculture, environment, and medicine, and INNs provide enhanced accuracy and efficiency in the classification and analysis of HSI data. In this context, it is evident that INNs contribute to further enhancing the potential of HSI technology, enabling more detailed and comprehensive analysis of data. The results of these studies underscore the reliability and effectiveness of INNs as tools for the analysis of HSI data, laying an important foundation for future research.