Development of cloud system based on deep learning for thermal image resolution enhancement

Thermal imaging systems provide non-contact temperature measurement and are harmless to human health. Therefore, it can be used safely on living things. Thermal cameras can be used in many sectors such as public security, health and defense that require the detection of temperature changes. However, the high cost of thermal cameras and their low edge detail information limit their usage. Thus, low-cost thermal cameras facilitate the use of thermal imaging in different areas. These thermal cameras can create low resolution thermal images with low detail information. Therefore, the need to improve the resolution of these low-resolution thermal images has emerged. Here, the use of super resolution techniques on these low quality images is of great importance. Super resolution is defined as the creation of a higher resolution image using a set of low resolution images; but deep learning-based applications in recent years, it would be more accurate to describe it as estimating a high resolution (ground truth) image from its low resolution (LR) counterpart. Although the concept of super resolution can be applied both in software and hardware, most of the studies carried out are on software-based super resolution applications. In recent years, hardware advances and developments in the field of deep learning have made deep learning algorithms more widely used in super resolution (SR) applications. Within the scope of the thesis study, super resolution applications were carried out by creating two different thermal databases. In addition, for super resolution applications, firstly, the "Thermal Super Resolution Generative Adversarial Networks (TSRGAN)" model based on generative adversarial networks (GAN) was designed, and then the TSRGAN+ model was developed in order to increase the success of this model. The first database used in super resolution applications was the thermal images of newborn babies (neonates) and were considered as high resolution (ground truth) images in the studies. Then, datasets consisting of low resolution images were obtained by down-scaling at certain ratios (1/2, 1/4, 1/8 and 1/16). The obtained results were evaluated with the image quality metrics peak signal to noise ratio (PSNR) and structural similarity index (SSIM). When PSNR and SSIM values are examined, it is seen that the developed models are more successful when compared to the state-of-the-art models. Also, when the TSRGAN+ model is compared with the TSRGAN model, PSNR values increased in the range of 1-1.5 dB, while SSIM values increased by 2-3%. In addition, a convolutional neural network-based (CNN-based) classifier model was designed to make task-based evaluation, and the applications were carried out to classify unhealthy-healthy babies. When the results were evaluated, it was observed that the classification success of super resolution images increased by about 9-11% compared to low resolution images. In addition, the TSRGAN+ model was found to be approximately 3% more successful than the TSRGAN model. In addition, the TSRGAN+ model has achieved approximately 3% more successful classification success than the TSRGAN model. The second database used within the scope of the thesis studies consists of thermal face images. This database was created by means of two different thermal cameras. Here, high-resolution (ground truth) images were obtained using a high-cost thermal imaging system, while low-resolution images were obtained through a low-cost thermal camera that can be attached to a smart phone. In super resolution applications, TSRGAN and TSRGAN+ models were used again. Here, it is aimed to avoid cost problems in thermal imaging projects by bringing the performance of a low-cost camera closer to that of a high-cost camera. When the obtained results were evaluated, it was observed that PSNR and SSIM values increased compared to the state-of-the-art models. In addition, the TSRGAN+ model was found to be approximately 0.5 dB more successful in PSNR and 5% more successful in SSIM compared to the TSRGAN model. In addition, a study was conducted to run the created super resolution system in the cloud environment. It is foreseen that super resolution techniques can be utilized in real time and actively through the cloud-based system created using an Android interface. Thanks to the cloud-based system, which can be easily accessed by different users and adapted to different applications, low-cost thermal cameras will be widely used in real-life applications. Keywords: Classification, Cloud computing, Convolutional neural networks, Deep learning, Generative adversarial networks, Super resolution, Thermal imaging