Elemination of ghosting effect using deep learning in high dynamic range image generation

Current digital imaging devices can only capture a small portion of the colors in the natural environment due to their limited capturing capabilities. Since the difference between the color levels of these images is small, these images are referred to as Low Dynamic Range (LDR). As dark or excessively bright areas can occur in LDR images, information about these areas can be lost. Combining differently exposed LDR images to obtain High Dynamic Range (HDR) images is often used to compensate for the loss of information in LDR images. Although this approach works well in scenes where the camera and the environment are stationary, in scenes with a moving camera and/or moving objects, it leads to a visual artifact called the ghost effect. This is caused by the displacement of the corresponding pixels in the merged poses. In this study, a two-stage method is proposed to remove the ghost effect in HDR image reconstruction. In the first stage of the proposed method, differently exposed LDR images are aligned to a selected reference image using a network that first performs homography based on the RANSAC algorithm and then performs motion estimation based on optical flow. In the second stage of the proposed method, a merging network is developed to generate an HDR image using three differently exposed input images and obtained aligned images. The merging network was trained and tested with the SIGG17 and ICCP19 datasets. The HDR images generated by the proposed method are compared quantitatively and qualitatively with the HDR images generated by a well-known method in this field. For quantitative evaluation, the PSNR and SSIM metrics are used, and better results are obtained than with the well-known method. The qualitative comparison shows that the visual results of the proposed method are very close to the target image, as the ghost effect is better removed.