A novel deep learning based approach for spatiotemporal image fusion

High temporal and spatial resolution plays an important role in applications such as disaster monitoring, monitoring of the growing cycle of plants, urban resource monitoring, and forest change detection. However, most publicly available remote sensing images don't have both high temporal and spatial resolution due to economic and technical reasons. This problem can be addressed by producing synthesized images with high spatial and temporal resolution using spatiotemporal image fusion. Convolutional Neural Networks (CNN) are useful models for spatiotemporal fusion, especially under strong temporal changes. However, existing CNN-based methods are not able to fully utilize hierarchical features due to the connections they contain. Convolutional layers with receptive fields of various sizes produce hierarchical features that can improve prediction performance. The goal of this thesis is to develop a CNN model that can fully utilize hierarchical features and extract them to overcome the limitations of the existing CNNs produced for spatiotemporal fusion. The network proposed within the scope of the thesis called Residual Dense Network for Spatiotemporal Fusion (STFRDN) is composed of residual dense blocks with local dense connections that effectively utilize the hierarchical features. Kansas dataset, which includes Sentinel-2 and Sentinel-3 image pairs with large resolution differences and strong temporal changes, was developed for the experiments. Based on both quantitative and qualitative evaluation, experiments revealed that the proposed STFRDN algorithm outperformed the Flexible Spatiotemporal DAta Fusion (FSDAF) 2.0, Reliable and Adaptive Spatiotemporal Data Fusion (RASDF), and DMNet methods in all of the test groups.