Motion block based video super resolution

ABSTRACT: A multi-frame super resolution process can be used for enhancing the resolution of video frames by employing the information of consecutive low-resolution frames taken from almost the same scene. Most of these super resolution algorithms are only suitable for global motion model. Nevertheless, if a local motion pattern such as movements of some objects happens between the low resolution frames a global motion model cannot provide efficient performance. Considering this problem, we propose a novel super resolution framework, where the moving and static regions in video frames are processed separately. Occlusion is another issue, which is not considered in most of the video super resolution processes. This problem occurs when a new object appears or an object disappears in the video frames. The proposed motion-block based super resolution method not only offers a local motion model but also deals with the occluded areas in a proper way. This thesis presents a new video super resolution technique, based on the motion and static areas of the low resolution video frames. In order to separate the motion and static blocks, a block motion estimation method is employed between a reference and its neighbouring frames. Among the motion blocks, the occluded blocks are identified using an adaptive threshold applied on each block individually. Structure-adaptive normalized convolution (SANC) reconstruction method is used to generate the high resolution static and motion blocks where discrete wavelet transform (DWT) based interpolation is used to produce the high resolution occluded blocks. The static and motion blocks are combined into a high resolution frame. Finally, a sharpening process is performed on the high resolution frame in order to generate the super resolved high resolution output frame. The experimental results show that the proposed technique generates significantly better qualitative visual results as well as higher quantitative PSNR and SSIM than the state of the art video super resolution algorithms. Keywords: Super resolution, resolution enhancement, multi-frame super resolution, video super resolution, motion estimation, local motion patterns. …………………………………………………………………………………………………………………………………………………………………………………………………………