Heart rate detection using physiological signs and its applications

In media and online platforms, the sharing of video and visual content has been increasing and they can be easily accessed. Artificial intelligence technology has made it easy to create fake video and visual content using a variety of algorithms. Such content poses serious security risks, such as blackmail, spreading false information, defamation, and creating political problems. This kind of manipulation can make people appear to be acting in ways they have never done or being in an environment they have never been in. The detection of fake contents are crucial since malicious content can be created with ease and without the need for programming experience. Although fake videos are created using different algorithms, they have biological signals similar to real video signals. However, there are many inconsistencies in fake videos, such as light differences, image compression, and the inadequacy of video creation models. Such reasons weaken the energy of biological signals in fake videos. A study was carried out to identify video forgeries by utilizing key physiological signal features, consistent with this methodology. 10 real and 10 fake videos lacking motion from the PURE dataset and 10 real and 10 fake videos from the UBFC dataset were examined. The regions under the forehead of the real video samples were altered to create fake videos. The initial focus of this thesis study is to obtain an accurate rPPG signal from facial videos. Facial region analysis with high performance was conducted to ensure accurate and robust signal extraction. rPPG signals were obtained by defining an orthogonal plane to the skin in the temporally normalized RGB space in different regions of the face. Heart rate detection was performed by applying the Short-Time Fourier Transform (STFT) to these rPPG signals. The correlation values between the heart rate signals of the real videos in the PURE dataset and the generated fake videos and the reference heart rate signals were calculated. Forgery analysis was carried out if the videos contained detectable heart rate signals. The forehead area, nose, cheeks, and regions under the forehead of the face were used to detect forgery in videos. By applying the Fast Fourier Transform (FFT) to rPPG signals in these regions, the highest energy magnitudes in the frequency spectrum for video samples were detected. Since the forehead region was not changed in the videos, it was chosen as a reference. The differences in energy magnitude between the forehead and cheeks, as well as between the forehead and cheeks, including the nose, and between the forehead and the area beneath the forehead, were calculated. Forgery analysis is not conducted if the differences calculated are lower than a specific threshold value. A forgery detection method is suggested in the videos if the computed differences exceed a specific threshold value.