Development of a deep learning based hybrid model to complete the missing part in the images

In this thesis, a hybrid model based on Convolutional Neural Network (CNN), Long Short-Term Memory (LSTM) and Autoencoder method were designed to predict missing parts of images. The estimation of missing parts in images is very important for images that have been interrupted, do not fit the cinema mode, and have an undesirable object. The study was worked on the Cifar-10 dataset. The autoencoder is used to compress the images in a meaningful way. Generative Adversarial Networks (GANs) method was used to control the data distributions of the compressed images and the distribution of the compressed images converged to the normal distribution. While the inputs of the CNN model are Cifar-10 images, the inputs of the LSTM model are compressed image data, which is the output of the Autoencoder model. Feature extraction was performed by both the CNN and LSTM model, and the extracted features were combined. These features were then given as input to a new LSTM and CNN model, and the missing parts of the images were estimated. CNN and LSTM generate two different forecasts. Root Mean Square Deviation (RMSE), Structural Similarity Index Calculation (SSIM), Generative Adversarial Networks Loss, and VGG (Visual Geometry Group, pre-trained model) loss were used as error metrics during the training of the model. The VGG loss represents the difference between the outputs of the previously trained VGG model. The results of the developed model were compared with the classical methods and shared in a table. While image processing methods are frequently used in the literature to predict missing parts of images, a hybrid model was created in this study. The hybrid model output consists of the weighted average of the LSTM output and the CNN output. The hybrid model output proved its suitability for this problem by getting the highest scores among the models tested on the RMSE, Signal-Noise Ratio (PSNR), SSIM and Fréchet Initial Distance (FID) metrics on the test dataset.