Kısmi kapanmaya uğramış insan eklemlerinin Bayesyen yaklaşımla kestirimi ve insan görüntüsü tamamlama uygulaması

Human pose estimation is a well-known computer vision task that has applications in the fields of surveillance, computerized outfit planners, and video special effects. State-of-the-art pose estimators are based on CNN structures and use visual features obtained from single images. However, occlusions are prevalent problems in the natural scenarios for this task, and the performances of CNN-based estimators degrade significantly under occlusion conditions. In this thesis, a novel Bayesian approach, BAKE: Bayesian Approach for occluded Keypoint Estimation, is presented to estimate the positions of occluded human joints in a given video sequence. This approach uses a well-known CNN-based pose estimator Openpose [1] to detect the visible human joints in a given image and then develops a Bayesian framework to complete missing pose elements. This approach can be evaluated as an alternative for 3D CNN structures in terms of embedding the information in time-dependent event sequences. In our case, the problem is ill-conditioned since it is in general not possible to complete the missing joints for an arbitrary occlusion on the human body accurately. However, it is possible to localize some missing joints in certain regions based on the apriori information on the human skeleton with a certain confidence. This apriori information is obtained from the previous frames of the video sequence. A statistical human body model is generated by defining the joint length and angle parameters. The parameters of the model are calculated from the non-occluded frames of the video sequence as the local information as well as a human pose database is utilized for obtaining the general joint statistics. Then, on a partially occluded video frame, body length and angle distribution are updated by using the visible joints. These length and angle distributions are used for the estimation of occluded joints. A new confidence score is also proposed. This confidence score is used to develop a hybrid technique which combines the predictions of the Openpose and the proposed method, BAKE. Several experiments are performed to compare the outputs of the Openpose, BAKE, and the hybrid approach. It is shown that BAKE outperforms Openpose in general and the hybrid method generates a slight improvement over the BAKE. In addition to the BAKE method, an inpainting method for the partially occluded human video frames is proposed. In this method, non-occluded images of the target person obtained from the video sequence are used with a 3D body reconstruction algorithm, SMPLify-x [2]. Image patches are transferred from non-occluded images to occluded parts after a matching process and the corresponding results are shown.